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PREFACE 


This  Lecture  Series  No.  100  on  the  subject  of  Methodology  for  Control  of  Life  Cycle 
Costs  for  Avionics  Systems  is  sponsored  by  the  Avionics  Panel  of  AGARD  and  implemented 
by  the  Consultant  and  Exchange  Programme. 

The  continually  increasing  costs  of  avionics  systems  during  acquisition  and  their  lifetime 
operation  is  a matter  of  grave  concern  to  the  NATO  family  of  nations.  The  NATO  Govern- 
ments need  greater  visibility  and  control  over  the  life  cycle  costs  of  any  weapon  or  avionic 
system. 

Fortunately  there  have  been  formulated  disciplined  methods  of  providing  such  visibility 
and  control  over  life  cycle  costs;  that  is  over  the  development  acquisition,  training,  operating 
and  support  and  Finally  disposal  costs. 

For  these  reasons  it  has  been  proposed  by  the  Avionics  Panel  to  sponsor  a Lecture  Series 
which  presents  the  basic  principles  of  Avionics  Systems  Cost  Analysis  in  a rapidly  changing 
technology  environment  and  gives  proven  methods  of  achieving  significant  costs  savings. 

The  Lecture  Series  covers  in  particular  the  following  subjects:  life  cycle  costing,  design 
to  cost,  technology  environment,  costing  of  software,  modelling  and  applications.  The  new 
trends  and  advantages  of  control  of  life  cycle  costs  for  avionics  systems  is  an  important  part 
of  the  Round  Table  Discussions  organised  in  the  two  locations  where  the  Series  is  presented. 
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Introduction  to  "Methodology  for  Control  of  Life  Cycle  Costs  for  Avionics  Systems 

1.  Introduction  Dr.  I.  J.  Gabelman,  Technical  Associar  i 

Advanced  technology  has  made  available  to  the  NATO  military  commander  an  array  of  highly  sophisti- 
cated, extremely  complex  systems  which  help  him  to  reach  his  operational  objectives.  Acquiring  this 
Increased  capability  however  has  been  costly s so  costly  that  it  presents  a significant  budgetary  problem 
to  the  member  nations  of  NATO.  The  Life  Cycle  Costs  (LCC),  defined  as  the  total  costs  of  acquiring,  op- 
erating and  supporting  a system  over  Its  lifetime,  has  come  under  careful  scrutiny.  Methodology  has 
been  evolved  which  enables  costs  of  currant  weapon  systems  to  be  reduced  and  costs  of  weapon  systems  now 
in  development  to  be  controlled. 

The  most  visible  costs  are  those  associated  with  procurement,  research  and  development,  test  and 
evaluation.  These  account  for  perhaps  one-third  of  the  total  LCC.  Operation  and  maintenance  and  man- 
power costs  are  roughly  the  other  two-thirds.  Cost  reduction  techniques  can  be  applied  by  everyone  who 
is  involved  and  contributes  to  the  fielding  of  a weapon  system  in  any  phase. 

Life  cycle  costs  can  be  lowered  in  many  ways.  In  the  development  and  acquisition  phase  of  a system 
acquisition  wherever  possible  objectives  should  be  limited-,  wherever  possible  commercial  products  parti- 
cularly those  with  high  production-volume  parts  should  be  used,-  simple  designs  should  be  employed  and 
the  data  required  should  be  minimized. 

Product  reliability  improvement  decreases  support  costs  in  the  field.  While  a great  deal  of 
research  and  development  is  taking  place  in  the  laboratories  to  improve  product  reliability  there  are 
several  approaches  that  can  be  employed  now  to  increase  system  reliability  and  decrease  LCC.  Simple 
proven  designs  should  be  used;  production  and  quality  control  should  be  improved  and  more  efficient  dev- 
elopment and  test  procedures  employed. 

In  the  area  of  logistic  support,  common  support  equipment  should  be  used;  standardization  should  be 
extensively  employed;  training  and  documentation  should  be  improved  and  commercial  resources  used. 

2.  Lecture  Series  Summary 

This  two-day  lecture  series  will  introduce  the  most  recent  developments  in  life  cycle  cost  method- 
ology for  avionics  systems.  The  speakers  represent  Canada,  the  United  Kingdom,  and  the  United  States, 
and  will  support  their  discussions  with  examples  drawn  from  current  experience  in  NATO  nations  in  the 
application  of  life  cycle  cost  programs  on  avionics  systems. 

Following  are  summaries  of  the  presentations: 

The  Life  Cycle  Cost  Concept  - A Management  Dr.  E.  N.  Dodson 

Tool  for  Planning  and  Control  General  Research,  Corp.,  U.S. 

Dr.  Dodson  will  explore  the  life  cycle  cost  concept  as  a management  tool.  He  will  discuss  the  prin- 
ciples upon  which  LCC  rests  and  the  procedures  used  in  its  implementation  in  the  development  and  acquisi- 
tion phase  of  system  procurement.  In  particular,  he  will  describe  the  elements  of  the  design- to-cost 
program,  the  need  for  parametric  cost  estimating  and  other  estimating  techniques  to  cope  with  rapid  tech- 
nological change,  and  the  elements  of  risk  analysis. 

Dr.  Dodson  will  illustrate  his  discussions  with  examples  drawn  from  data  processing  systems  in 
avionics  environments.  These  examples  will  be  considered  from  the  point  of  view  of  hardware  and  software. 

The  Development  and  Implementation  of  T.  D.  Kiang 

Life  Cycle  Cost  Methodology  Bell-Northern  Research,  Canada 

Mr.  Kiang  will  discuss  the  methodology  developed  for  the  Canadian  Department  of  National  Defence  for 
carrying  out  comparative  engineering  evaluations  as  an  aid  to  management  decision-making.  This  methodology 
is  unique  in  that  it  relates  the  life  cycle  cost  of  a system  tc  a system  effectiveness  parameter,  namely 
availability.  The  generalized  life  cycle  management  cost  model  can  analyse  a system  of  one  or  more  prime 
equipments  situated  at  one  or  more  locations,  and  takes  into  account  the  maintenance  and  logistics  sup- 
port provided.  Applications  of  the  model  will  be  illustrated  by  examples  from  a study  carried  out  on  the 
Micro-Electronic  Airborne  TACAN  equipment.  Mr.  Kiang  will  discuss  the  decision  flow  involved  in  implem- 
enting this  LCC  methodology  in  the  Canadian  defence  environments,  and  will  describe  some  of  the  implica- 
tions and  management  problems  in  LCC  implementation. 

Recent  Experience  in  the  Mr.  J.  J.  Naresky 

Application  of  LCC  Models  Rome  Air  Development  Center,  U.S. 

Mr.  Naresky  will  describe  recent  experience  in  the  application  of  some  of  the  LCC  models  currently 
used  in  the  U.S.  military  environment,  including  Acquisition  LCC  models  and  Operation  and  Logistics 
Support  Models.  He  will  present  material  from  some  recent  RADC  contracts  concerning  cost-estimating 
and  relationships,  LCC  trade-offs,  and  RIV  incentives.  He  will  also  discuss  the  derivation  of  con- 
fidence levels  in  estimating  LCC  costs. 
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Life  Cycle  Cost  Producrement 
In  The  UK 


Mr.  P.  G.  Reich 
Ministry  of  Defence,  U.K. 


Mr.  Reich  will  describe  UK  experience  In  the  acquisition  and  procurement  of  avionics  system  using 
LCC  methodology.  He  will  cover  the  source  selection  and  evaluation  process  In  LCC  procurement,  and 
detail  typical  LCC  contract  provisions.  Mr.  Reich  will  discuss  the  significance  of  data  requirements 
and  data  sources  In  I£C  procurement,  as  well  as  data  availability,  Interpretation,  and  applicability. 
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LIFE  CYCLE  COST  ANALYSIS: 
CONCEPTS  AND  PROCEDURES 

Edward  N.  Dodson 
General  Research  Corporation 
P.0.  Box  6770 

Santa  Barbara,  California  93111 


SUMMARY 

Limited  budgets  and  the  growing  costs  of  high-technology  have  prompted  great  interest  in  cost  estimation, 
a field  in  which  General  Research  Corporation  (GRC)  has  been  deeply  involved  for  over  ten  years.  We  have  been 
particularly  concerned  with  parametric  estimating  techniques — techniques  which  are  most  suitable  during  the 
early  phases  of  an  equipment's  "life-cycle."  GRC  has  conducted  methodological  studies  for  the  Assistant 
Secretary  of  Defense,  each  of  the  armed  services,  the  National  Aeronautics  and  Space  Administration,  and  a 
number  of  other  government  agencies. 

This  paper  is  drawn  from  these  various  studies  and  is  intended  as  a basic  exposition  of  the  parametric 
method  of  cost  analysis,  especially  as  it  applies  to  modern  high-technology  systems.  Hopefully,  the  reader 
will  find  some  well-founded  and  practical  guidance  to  the  estimation  of  costs  for  new  systems. 

1.  INTRODUCTION 

An  evaluation  of  overall  program  costs  and  benefits  is  a fundamental  element  of  good  management.  Such 
evaluations  are  required  in  order  (1)  to  decide  whether  to  undertake  a particular  program,  (2)  to  choose 
among  alternative  program  designs,  and  (3)  to  establish  a specific  project  plan  of  resource  commitments  and 
anticipated  returns.  The  general  literature,  and  indeed  the  daily  news,  are  replete  with  examples  of  pro- 
jects for  which  the  initial  cost  estimates  proved  to  be  grossly  in  error.  In  many  instances  programs 
afflicted  with  overruns  have  been  cancelled  after  considerable  expenditure;  more  often,  quantities  are  cut 
back  or  performance  is  compromised.  In  either  case  there  is  a patent  waste  of  valuable  resources. 

The  parametric  method  of  estimating  costs  is  particularly  well-suited  for  the  early  phases  of  a pro- 
gram's "life-cycle" — the  time  when  comparatively  few  details  about  the  eventual  equipments  are  known,  yet 
many  of  the  important  program  decisions  must  be  made.  Both  the  General  Accounting  Office  (GAO)  and  the 
Department  of  Defense  (DoD)  have  indicated  that  by  the  time  a system  completes  the  Concept  Formulation  stage 
(DSARC  I in  DoD  parlance)  70%  of  the  life-cycle  cost  has  been  determined,  i.e.,  has  been  fixed  by  the  choices 
made  to  that  point  in  the  life  cycle.  By  completion  of  the  System  Definition  stage  (DSARC  II),  85%  of  the 
cost  is  determined. 

.1  * 

A recent  study  within  the  Department  of  Defense  elaborates  upon  the  merit  of  the  parametric  approach: 

...estimates  for  new  weapon  systems  acquisition  costs  are  either  derived  from  detailed, 
grass  root  calculations  (the  industrial  engineering  approach)  or  based  on  relationships 
between  more  aggregated  components  of  system  cost  and  the  physical  and/or  performance 
characterist ics  of  the  system.  These  relationships  should  be  derived  from  cost  his- 
tories on  prior  programs.  The  latter  method  is  often  called  the  parametric  approach. 

It  is  clear  that,  during  the  early  phases  of  the  acquisition  process,  only  limited 
design  Information  is  available  and  considerable  uncertainty  surrounds  both  this  in- 
formation and  whatever  planning  data  is  available  on  how  the  new  system  will  be  devel- 
oped and  produced.  Nonetheless,  cost  estimates  must  be  made.  Both  the  fact  of  limited 
and  uncertai  information  on  which  to  base  estimates,  and  the  use  to  be  made  of  these 
cost  estimates,  strongly  suggest  the  employment  of  parametric  estimating  procedures. 

The  parametric  approach  is  particularly  suited  to  making  estimates  based  on  limited 
physical  and  performance  information. 

For  most  new  systems,  the  parametric  approach  is  the  only  method  that  can  be  used  to 
make  an  estimate  from  the  limited  information  available  during  concept  formulation, 
i.e.,  when  only  mission  and  performance  envelopes  are  defined.  Only  subsequently 
when  detailed  contractor  proposals  are  being  prepared  can  the  industrial  engineering 
procedures  be  applied.  Furthermore,  parametric  methods  provide  the  analyst  with  an 
inexpensive  means  of  examining  the  impact  on  cost  of  a variety  of  changes  in  system 
performance  requirements — information  of  particular  importance  during  the  early  phases 
of  the  development  and  planning  processes. 

In  short,  the  parametric  approach  is  based  upon  aggregate  relationships  between  cost  and  the  physical/ 
performance  characteristics  of  the  system  under  study.  These  relationships  are  derived  from  related  his- 
torical data,  generally  following  the  principles  of  statistical  inference. 

This  paper  outlines  the  general  principles  and  procedures  of  parametric  cost  analysis.  In  doing  so  it 
draws  upon  a number  of  earlier  reports  and  documents4-”^  representing  studies  conducted  for  many  federal 
agencies.  While  most  of  these  have  been  within  the  Department  of  Defense  (including  each  of  the  services  and 
various  elements  within  the  Office  of  the  Secretary  of  Defense),  we  have  also  developed  costing  procedures 
and  est imat ing- relationships  for  the  Environmental  Protection  Agency,  the  Department  of  Housing  and  Urban 
Development,  NASA,  and  others. 


* 


While  many  of  the  discussions  and  examples  in  this  paper  emphasize  military  systems,  the  methods  and  pro- 
cedures are  applicable  to  high-technology  equipment  in  such  diverse  fields  as  transportation,  commercial 
electronics,  etc. 


The  discussions  in  this  paper  consider  costs  sole!,  in  terms  of  direct  dollar  outlays,  ignoring  social, 
environmental,  and  other  costs  external  to  the  conventional  measures  of  dollar  costs  (although  the  general 
principles  are  appropriate  and  have  been  applied  to  these  and  other  factors). 

Most  of  our  illustrations  concern  the  cost  to  the  government  of  work  done  by  contractors  during  the 
production  phase  of  the  material  life  cycle,  reflecting  thereby  the  actual  emphasis  (and  state-of-the-art) 
in  the  cost  analysis  community.  While  the  principles  of  statistical  inference  are  applicable  to  other  ele- 
ments of  costs,  the  difficulty  of  acquiring  data  and  in  establishing  cost-determining  relationships  have 
limited  the  application  of  parametric  techniques  to  phases  such  as  Research  and  Development,  and  operation 
in  the  field. 

In  Sec.  2 the  basic  elements,  or  "steps,"  of  parametric  cost  analysis  are  discussed  and  illustrated. 
While  these  steps  are  set  forth  in  separate  and  sequential  paragraphs,  it  should  be  emphasized  that  there 
are  numerous  interrelations  and  "feedback  loops"  among  them  (as  illustrated  in  simplified  form  in  Fig.  1.1). 
For  example,  one  of  the  first  steps  entails  developing  a formally  structured  table  of  the  individual  cost 
elements  to  be  considered.  Ideally,  the  organization,  detail,  and  definition  of  these  cost  elements  could 
be  set  forth  at  the  outset.  In  practice,  however,  such  classifications  often  must  be  revised  to  reflect  the 
format  of  the  cost  accounting  information  obtained  during  the  data  collection  phase. 

Similarly,  the  formulation  of  cost  hypotheses  and  their  statistical  evaluations  are  often  conducted  in 
a series  of  iterative  steps.  Data  collection,  of  course,  is  also  closely  interrelated  with  the  formulation 
of  hypotheses  and  the  evaluation  of  Cost  Estimating  Relationships  (CERs) . 

Section  3 discusses  several  refinements  and  extensions  of  parametric  cost  analysis. 

2.  PARAMETRIC  COST  ANALYSIS 

2.1  Objectives,  Scope,  and  Choice  of  Approach 

The  first  step  in  parametric  cost  analysis  (and  any  other  analytic  effort,  for  that  matter)  is  to  estab 
lish  the  objectives  of  the  analysis  and  its  scope.  Three  common  objectives  are: 

1.  To  compare  estimates  of  the  total  costs  of  a particular  program  with  program  benefits  before 
deciding  whether  the  program  should  be  initiated. 

2.  To  estimate  total  costs  so  that  future  budgets  and  financial  requirements  can  be  established. 

3.  To  compare  costs  among  competing  alternatives. 

While  the  analyst  may  of  course  be  required  to  meet  any  or  all  of  these  objectives,  each  poses  some  dis- 
tinctive analytic  requirements.  If  the  objective  of  the  analysis  is  only  to  make  cost  comparisons  among 
competing  systems,  its  scope  may  be  limited  to  estimating  costs  only  for  those  items  and  characteristics 
which  differ  among  the  candidate  systems.  For  other  purposes,  all  the  costs  associated  with  the  program(s) 
under  study  must  be  taken  into  account.* 

Another  distinction  among  these  objectives  pertains  to  the  use  of  price-level  indices.  Cost-benefit 
analyses  should  be  made  in  "constant"  dollars  (to  the  extent  that  benefits  and  costs  are  measured  in  dollars 
while  future  budgetary  planning  normally  requires  "current"  dollars.  In  short,  program  evaluations  are 
generally  conducted  in  terms  of  constant  dollars,  while  budgetary  planning  requires  time-phasing  of  expen- 
ditures in  current  (or  "then-Year")  dollars. 

Defining  problem  scope  requires  decisions  as  to  the  degree  of  general  1 tv  desired,  and  this  in  turn 
touches  upon  the  suitability  of  the  parametric  approach.  By  degree  of  generality  we  mean  such  questions  as 

...Is  the  analysis  to  provide  estimating  relationships  for  a general  class  of  equipments 
(such  as  tactical  missiles)?  Or  is  the  problem  more  narrowly  defined  (e.g.,  air-to-air 
tactical  missiles,  or  air-to-air  tactical  missiles  using  radar  guidance)?  Or,  is  the 
estimate  for  a specific  missile  whose  characteristics  are  fully  defined? 

If  the  task  is  to  estimate  costs  of  an  item  whose  design  is  (1)  specified  in  full  detail,  and  (2)  closely 
related  to  existing  design  and  production  techniques,  the  industrial  engineering  approach  is  probably  the 
better  choice  to  establish  a definitive  estimate.  An  independent  parametric  estimate  may  still  be  useful 
as  a check  of  the  more  detailed  estimate. 

2.2  Cost  Chart  of  Accounts 

The  next  step  in  the  analysis  is  to  develop  a formally  structured  table  of  the  cost  elements  to  be 
examined.  The  purpose  of  this  "structure,"  called  the  Cost  Chart  of  Accounts,  is  to  ensure  that  (1)  all 
costs  are  taken  into  account,  (2)  none  are  double-counted,  and  (3)  individual  cost  elements  are  consis- 
tently and  clearly  defined.  The  Cost  Chart  of  Accounts  is  a basic  tool  for  organizing  the  analysis  in 
keeping  with  its  prescribed  objectives  and  scope.  The  importance  of  this  tool  in  defining  the  individual 
cost  elements  cannot  be  overstated;  more  often  than  not,  differences  in  cost  estimates  stem  from  differences 


There  are  several  issues  to  be  resolved  in  defining  "all"  of  the  costs;  these  are  discussed  in  Sec.  2.2. 

Much  of  the  debate  about  use  of  inflation  indices  seems  to  reflect  a degree  of  confusion  of  these  distinct 
objectives.  "Constant"  dollars  represent  costs  expressed  in  terms  of  a specific  year.  Costs  from  other 
years  ("current"  dollars)  are  normalized  to  the  base  year  (made  "constant")  bv  use  of  price  indices  (see 
discussion  in  Sec.  2.5). 


in  defining  what  is  included  in  the  estimate.  For  example,  does  hardware  production  cost  include  the  costs 
! manufacturing  engineering,  quality  control,  or  documentation?  Are  non-recurring  costs  allocated  pro  rata 
to  the  unit  cost  of  hardware?  Are  autopilot  costs  included  in  missile  guidance  or  in  missile  control? 

These  representative  issues  are  resolved  in  a carefully  structured  Chart  of  Accounts. 

There  .ire  several  ways  to  structure  the  Cost  Chart  of  Accounts.  At  CRC  we  have  generally  used  a 
st raight forward , two-dimensional  matrix.  One  axis,  called  the  Work  Breakdown  Structure,  defines  the  "end 
items"  (e.g.,  laser  optica)  while  the  second  indicates  various  elements  of  the  system  life-cycle  (e.g., 
initial  tooling). 

A Work  Breakdown  Structure  is  an  organized  array  which  identifies  hardware,  software,  and  services  pro- 
duced or  performed  during  the  life  of  a weapon  system  program.  An  example  of  a hardware  item  is  a missile 
guidance  system;  an  example  of  a service  is  system  test  and  evaluation.  The  DoD  document  establishing 
criteria  for  the  preparation  and  employment  of  work  breakdown  structures  used  in  the  acquisition  of  defense 
material  items  is  Military  Standard  881.  This  document,  intended  for  use  by  both  contractors  and  DoD 
organizations,  defines  a WBS  as: 

...a  product-oriented  family  tree  composed  of  hardware,  software,  services,  and  other  work 
tasks  which  result  from  project  engineering  efforts  during  the  development  and  production 
of  a defense  material  item,  and  which  completely  defines  the  product/program.  A WBS  dis- 
plays and  defines  the  product (s)  to  be  developed  or  produces  and  relates  the  elements  of 
work  to  be  accomplished  to  each  other  and  to  the  erui  product. 

Again  quoting  from  MIL-STD  881,  the  purpose  of  a WBS  is  to  provide  a framework  that  facilitates: 

1.  Effective  management  and  technical  base  for  planning  and  assigning  management  and  technical 
responsibilities  within  government  offices  responsible  for  the  acquisition  of  defense  material 
items  and  contractors  furnishing  those  items. 

2.  Consistent  control  and  reporting  of  the  progress  and  status  of  engineering  and  other  con- 
tractor efforts  throughout  the  development  and  production  of  defense  material  items. 

Intended  as  a management  planning  and  control  device,  it  is  not,  therefore,  necessarily  intended  to  re- 
present the  best  format  for  cost  analyses.  The  WBS  formats  provided  in  MIL-STD  881  may  be  and  often  are 
modified  by  government  and  contractor  organizations.  Indeed,  MIL  STD  881  recognizes  the  need  for  flexi- 
bility in  developing  a WBS: 

When  the  prescribed  elements  and  definitions  are  not  applicable  (because  of  an  item's 
unique  configuration,  activities,  or  other  requirements  or  when  additional  WBS 
element  selection  is  needed  or  desired  beyond  the  summary  WBS)  additional  or  sub- 
stitute elements,  properly  def ined , . . .may  be  used. 

Variations  in  WBS  format  and  in  the  definition  of  each  item  in  the  format  tend  to  be  a major  problem 
to  the  cost  analyst  when  working  with  historical  data.  These  problems  result  from  tailoring  the  WBS  to 
contractor  or  government  accounting  systems  or  the  needs  of  a particular  program  manager.  Consequently, 
there  may  be  as  many  formats  and  definitions  as  there  are  programs.  Furthermore,  there  may  be  more  than 
one  WBS  used  in  different  reports  for  a single  program  depending  on  the  purpose  of  the  report  or  the  level 
in  the  government /Indus try  hierarchy  at  which  the  report  is  to  be  used. 

Cost  analysts  who  are  concerned  with  new  systems  will  generally  be  required  to  develop  their  own  WBS 
formats.  The  organization  of  subsystems,  equipments,  and  components  should  be  chosen  to  reflect  the  general 
functional  organization  of  the  equipment  under  study.  Table  2.1  illustrates  the  WBS  developed  in  a recent 
GRC  study  of  high-energy  laser  systems.  This  WBS  also  reflects  some  of  the  special  requirements  which 
characterize  most  cost  studies.  The  laser  study  included  a variety  of  laser  types  and  modes  of  operation, 
and  the  WBS  was  devised  to  permit  flexibility  in  system  devices  within  the  same  WBS  framework. 

Life  Cycle  Cost  Categories 

The  second  basic  dimension  of  the  Cost  Chart  of  Accounts  addresses  the  major  phases  of  the  "life 
cycle,"  i.e.,  the  sequential  activities  in  the  history  of  a program  which  cover  its  inception  through  final 


Some  agencies  use  a three-dimensional  accounting  structure.  One  such  structure  includes  (1)  Areas  of 
Activity,  (2)  Subdivisions  of  Work,  and  (3)  Elements  of  Cost.  Areas  of  Activity  are  the  end-items, 
and  thus  correspond  to  the  Work  Breakdown  Structure.  Subdivisions  of  Work  indicate  the  type  of  effort, 
such  as  Design,  Management,  Documentation,  etc.,  while  the  Elements  of  Cost  break-out  the  individual 
skills  (technician,  engineer)  and  cost  categories  of  material,  overhead,  subcontract,  etc. 

In  our  two-dimensional  matrix  the  Elements  of  Cost  are  not  shown — the  detail  of  types  of  labor  and  other 
resources  are  not  broken-out  separately.  In  general,  the  parametric  method  is  not  used  to  estimate  costs 
at  such  a fine  level  of  detail.  Where  necessary,  some  "rules-of-thumb"  can  be  used  to  apportion  aggregated 
costs  among  these  detailed  elements. 

The  basic  laser  types  included  gas  dynamic  lasers,  chemical  lasers,  and  electric  discharge  lasers.  The 
different  operating  modes  included  pulse  or  continuous-wave  operation,  and  open-  or  closed-cycle  operations. 
As  far  as  the  WBS  is  concerned,  the  difference  among  these  alternatives  is  reflected  by  the  inclusion  or 
exclusion  of  particular  elements.  The  basic  WBS  includes  all  elements  which  might  be  used;  the  cost 
analyst  must  choose  those  suitable  for  his  particular  problem. 
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replacement.  The  Institutions  and  procedures  which  define  these  life  cycle  phases  within  the  Defense  De- 
partment and  NASA  have  been  evolving  over  the  past  several  years.  Current  Defense  Department  procedures 
are  summarized  in  Fig.  2.1.  Basically  they  account  for  the  following  major  phases,  or  cost  categories: 

Research  and  Development  - This  category  includes  those  expenses  which  occur  within  the  Vali- 
dation and  Full  Scale  Development  Phases  of  the  acquisition  cycle.  Included  are  all  costs  for 
the  engineering  design,  analysis,  development,  and  testing  which  are  associated  with  the  R&D 
effort  directed  to  the  elements  of  the  Work  Breakdown  Structure.  Normally  these  costs  are 
funded  from  the  RDTE  Appropriation,  but  other  appropriations  may  be  required  to  complete  the 
R&D  phase  of  the  acquisition  process. 

Investment  Non-Recurring  - This  category  includes  those  non-recurring  expenses  necessary  to 
acquire  a production  capability  which  are  directed  to  the  elements  of  the  work  breakdown 
structure.  They  are  one-time  expenses  or  capital  investments  and  include  the  cost  for 
Initial  Production  Facilities  (IPF),  as  well  as  any  other  one-time  cost  associated  with 
obtaining  a production  capability.  Normally  these  costs  are  supported  by  the  procurement 
appropriations,  but  other  appropriations  may  be  required  within  the  investment  phase  of 
the  acquisition  process. 

Investment  Recurring  - This  category  includes  those  expenses  which  are  of  a continuing  nature 
in  the  production  of  a weapon/support  system.  It  includes  all  costs  for  the  Low  Rate  Initial 
Production  (LRIP)  and  Full  Scale  Production  (FSP)  of  the  major  system  equipment,  as  well  as 
the  continuing  expenses  for  the  recurring  investment  effort  directed  to  the  elements  of  the 
Work  Breakdown  Structure.  Normally  these  costs  are  supported  by  the  procurement  appropria- 
tions, but  other  appropriations  (e.g.,  MPA  in  the  Army)  may  be  required  in  the  investment 
phase  of  the  acquisition  process.  Also  included  are  the  expenses  incurred  in  delivering 
the  weapon/support  system  or  its  components  to  its  destination,  all  engineering  changes,  and 
modifications  (a  separate  budget  activity)  occurring  after  acceptance  but  while  the  system 
is  still  in  production. 

Operating  Cost  - This  category  includes  those  expenses  resulting  from  the  operation,  modi- 
fication, maintenance,  consumption,  and  disposal  of  materials  and  supplies  for  a weapon/ 
support  system  after  acceptance  into  the  inventory. 

There  are  both  contractor  and  government  ("in-house")  costs  in  each  of  these  life-cycle  phases.  Typically, 
the  major  efforts  of  development  and  production  are  undertaken  by  a contractor,  with  the  government  effort 
including  management,  some  documentation  and  training  activity,  and  various  logistics  activities.  Within 
these  broad  categories  there  are  numerous  individual  cost  elements  which  account  for  specific  activities 
within,  say,  R&D  (see  examples  given  in  the  horizontal  axis  of  Table  2.1).  The  specific  titles  and  meaning 
of  these  elements  differ  among  the  military  services,  NASA,  and  other  agencies  reflecting  the  different  pro- 
cedures which  are  followed  in  developing,  producing,  and  operating  various  systems  and  equipments. 

Government  costs  are  generally  estimated  as  a small  percent  of  total  contractor  effort  (based  upon 
the  argument  that  this  management  activity  will  scale  with  the  overall  magnitude  of  the  contracted  pro- 
gram). In  some  studies  government  costs  are  estimated  as  a "flat  rate,"  e.g.,  $50,000  per  year  for  main- 
taining a supply  catalog. 

Operating  and  Support  are  largely  all-government  costs  (replenishment  spares  and  some  other  expenditures 
may  be  contractor-incurred) , and  are  seldom  estimated  with  equations  based  upon  performance  or  physical 
characteristics.  While  in  principle  they  could  be,  the  general  practice  is  to  rely  upon  manpower  inputs 
from  military  commands,  with  maintenance,  spares,  modif ications , and  other  O&S  costs  estimated  as  a factor 
of  production  costs.  As  the  state-of-the-art  in  cost  analysis  improves,  more  of  these  O&S  cost  elements 
will  be  estimated  parametrically. 

Level  of  Disaggregation 

As  illustrated  in  Table  2.1,  the  Cost  Chart  of  Accounts  includes  a hierarchy  of  detail,  with  success- 
ively finer  levels  of  detail  (or  "levels  of  indenture")  being  defined  within  each  major  end-time  and  life 
cycle  category.  One  issue  to  be  resolved  by  the  analyst  is  the  level  of  detail  at  which  individual  cost- 
estimating relationships  will  be  developed,  i.e.,  the  extent  to  which  the  systems  and  equipments  are  to  be 
disaggregated.  Several  factors  bear  upon  this  decision: 

• The  objective  of  the  analysis:  If  the  goal  of  the  analyst  is  to  identify  cost  differences 
among  individual  components  of  a system,  then  he  should  do  his  work  at  a level  of  detail 
which  is  fine  enough  to  reflect  the  distinctions  among  the  components . 

• The  availability  of  data:  The  shortage  of  relevant  data  at  a comparatively  high  level  of 
detail  often  prompts  a further  disaggregation  of  the  system  under  study  in  order  to  draw 
upon  a wider  base  of  analogous  subsystems  and  components.  To  illustrate — consider  the 
development  of  a CER  for  an  airborne  phased-array  radar.  There  have  been  very  few  of 
these  systems  actually  developed.  However  if  the  radar  is  disaggregated  to  a finer  level 
of  detail,  such  as  antenna,  transmitter,  receiver,  signal  processor,  power  supply,  etc., 
the  analyst  can  take  advantage  of  the  fact  that  the  transmitter,  power  supply,  and  other 
system  elements  are  actually  quite  similar  to  other  types  of  radars  and — in  many  cases--to 
other  equipments,  such  as  communcations  sets,  ECM  equipment,  etc.  This  permits  use  of  a 
larger  data  base  with  consequent  improvements  in  accuracy. 


"Appropriations"  refer  to  the  organization  of  budgetary  accounts  as  they  are  enacted  by  the  Congress. 
They  represent  a different  means  of  classifying  expenditures. 
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Tills  process  of  disaggregat  Ion  to  draw  upon  a larger  related  data  base  requires  special 
care  and  technical  understanding  on  the  part  of  the  analyst  to  ensure  that  the  "analogous 
subsystems"  are  truly  relevant  as  far  as  the  underlying  determinants  of  cost  are  con- 
cerned. If  his  chosen  analogs  are  really  different  In  terms  of  the  functional  relation- 
ships with  cost,  then  the  cost  model  will  be  mlsspecifled  and  the  results  will  be  erro- 
neous . 


I 


• Major  differences  in  uncertainty:  one  motivation  for  disaggregated  estimates  Is  to 
clearly  Identify  key  uncertainties.  For  example  we  have  generally  disaggregated  array- 
radar  estimates  in  order  to  isolate  the  uncertainties  associated  with  phase-shifter 
costs.  Similarly,  it  is  desirable  to  disaggregate  missile  costs  in  order  to  separate 
the  comparatively  uncertain  guidance  estimates  from  the  more  confident  estimates  for 
propulsion  and  airframe. 

• The  potential  for  omitting  key  cost  elements:  one  hazard  of  detailed  or  disaggregated  cost 
elements  is  that  significant  integration  and  other  system-level  costs  may  be  overlooked. 

The  analyst  must  be  careful  to  ensure  that  he  has  accounted  for  costs  which  may  not  be 
identifiable  with  particular  subdivisions  of  the  hardware,  but  which  are  nevertheless 

part  of  the  overall  system's  cost.  Thus,  for  example,  there  are  various  management  and 
integration  costs  identified  at  the  subsystem  and  system  level  in  Table  2.1. 

• Burdens  upon  the  user:  Another  consequence  of  developing  CERs  at  a highly  disaggregated 
level  of  detail  is  a formidable  number  of  individual  estimating  equations,  and  a resul- 
tant list  of  input  parameters  which  may  be  impracticable  for  most  users. 

The  ultimate  level  of  detail  is  invariably  a compromise  among  divergent  considerations.  It  has  been  our 
experience  that  cost  studies  of  high  technology  systems  must  be  conducted  at  increasingly  finer  levels 
of  detail.  In  this  sense,  the  sharp  distinctions  between  the  parametric  method  and  the  "industrial 
engineering"  approach  are  becoming  much  Less  pronounced. 

The  preceding  steps  have  established  a framework  which  defines  the  scope  and  detail  of  the  analysis. 

As  noted,  the  specific  elements  of  the  Cost  Chart  of  Accounts  are  invariably  modified  as  the  analysis 
proceeds — largely  because  of  the  need  to  adapt  the  cost  format  to  the  availability  of  data.  The  next 
task  of  the  analyst  is  to  develop  an  estimating  relationship  for  each  element  of  the  Chart  of  Accounts. 

2.3  Formulating  a Cost  Hypothesis 

The  principle  technique  used  in  parametric  cost  analysis  is  that  of  statistical  inference  using  the 
procedures  of  multiple  regression.  There  are  numerous  texts  which  develop  the  rigorous  principles  and 
underlying  theory  of  statistical  inference  (see  e.g..  Refs.  10-12).  Particular  emphasis  should  be  given 
to  the  principle  that  a specific  functional  relationship  between  independent  variables  (cost  "drivers") 
and  a dependent  variable  (cost)  should  be  hypothesized  on  the  basis  of  a technical  understanding  of  the 
process  under  study.  Then,  statistical  procedures  may  be  employed  to  verify  the  postulated  CER. 

In  practice,  however,  statistical  procedures  are  often  used  to  identify  the  independent  variables 
and  to  specify  the  functional  form  of  the  estimating  equation.  We  feel  this  procedure  is  incorrect, 
although  it  is  difficult  to  be  absolutely  rigorous  on  this  point.  Parametric  cost  analysis  is  often 
exploratory  and  there  is  invariably  some  tr ial-and-error  in  finding  a defensible  CER.  Thus,  a search 
(using  statistical  selection  criteria)  among  a limited  number  of  candidate  hypotheses  is  not  necessarily 
to  be  faulted.  An  understanding  of  the  underlying  relationships  between  dependent  and  independent  variables 
may  be  achieved  only  after  statistical  indicators  have  pointed  the  way.  But  there  must  be  an  eventual  logic 
and  rationale  for  the  engineering  and  economic  relationships  which  underlie  each  CER.  In  short,  the  cost 
analyst  must  have  an  understanding  of  the  process  he  is  studying  (e.g.,  development  of  a computer)  which 
approaches  the  knowledge  expected  of  an  engineer. 

Without  this  underlying  logic,  the  CER  can  be  very  misleading,  no  matter  how  persuasive  the  statis- 
tical indicators  may  appear.  Without  insight  and  care,  statistical  procedures  can  be  merely  an  organized 
way  of  going  wrong  with  confidence. 

With  this  preamble,  the  next  step  is  the  development  of  cost  hypotheses  (although,  as  noted,  this  step 
generally  proceeds  hand-in-hand  with  data  collection  and  statistical  evaluation). 

In  general,  the  analyst  strives  to  identify  a cause-and-ef feet  relationship  between  selected  indepen- 
dent and  dependent  variables.  However,  in  practice,  the  actual  relationship  among  the  identified  variables 
is  often  merely  one  of  statistical  correlation.  Aside  from  spurious  cor re lat ions , the  dependent  and  "in- 
dependent" variables  may  both  be  the  effect  of  some  common  unidentified  or  unstated  cause.  For  example, 
weight  and  input  power  are  often  observed  to  have  high  correlation  with  electronics  costs  and  are  used  bv 
some  analysts  as  independent  variables.  In  many  cases  these  variables  are  common  indicators  of  more  bast, 
causal  factors  (such  as  functional  complexity). 

For  purposes  of  prediction  this  statistical  association  may  be  sufficient,  1 e.,  the  analyst  need  not 
establish  an  actual  cause-and-ef feet  relationship,  as  long  as  the  same  statistical  relationship  will  he 
applicable  in  the  forecast  period  (see  discussion  on  the  following  page  about  "aval lab i 1 ity") . However 
the  analyst  should  be  aware  of  these  relationships  among  the  variables,  and  recognize  that  a different 
model  must  be  developed  for  tradeoff  analysis. 

There  are  two  main  elements  in  the  formulation  of  a CER:  (1)  selecting  the  candidate  independent 
variables,  or  parameters  of  the  CER,  and  (2)  selecting  a functional  form  for  the  CER.  Each  requires  a 


2.3.1  Selecting  Candidate  Parameters 

The  objective  of  this  sub-step  Is  to  Identify  which  descriptive  parameters  are  most  likely  to  in- 
fluence the  particular  cost  being  analyzed.  Typically,  for  production  costs,  the  analyst  will  seek  para- 
meters which  govern  subsystem  size  or  complexity.  For  example,  transmitter  power,  computer  speed,  and 
antenna  diameter  are  conventional  parameters  used  in  CKRs  for  these  respective  subsystems.  Other  parameters 
which  help  explain  variations  in  cost  also  may  be  identified;  e.g.,  variations  in  cost  for  a given  trans- 
mitter power  would  be  expected  as  a function  of  frequency.  These  parameters  may  be  further  modified  to 
reflect  different  design  options,  e.g.,  solid  state  versus  tube  designs. 

Tills  point  prompts  some  additional  comments  about  the  identification  of  cost-determining  parameters. 
The  result  of  most  systems  analysis  studies  are  expressed  in  terms  of  system  performance  measures,  such  as 
aircraft  range,  radar  detection  range,  communcat ions  bit  rates....  It  would  be  desirable  to  have  cost  re- 
lationships expressed  in  these  same  dimensions.  However,  to  achieve  various  specified  performance-levels, 
there  are  typically  a number  of  design  options,  each  of  which  Impacts  directly  upon  cost.  These  relation- 
ships between  performance,  design,  and  cost  are  summarized  below: 


Design  //I  ■ 


Performance : 


-Design  //2‘ 


With  the  passage  of  time,  technological  change  can  affect  both  the  performance-design  and  design-cost 
relationships.  For  systems  characterized  by  diverse  and  rapidly  changing  technologies,  the  use  of  a 
performance  parameter  in  a cost  equation  incurs  the  risk  of  considerable  variability.  In  these  instances 
the  analyst  should  carefully  limit  the  performance-cost  relationship  to  particular  designs  and  technological 
choices  (this  point  is  discussed  further  in  Sec.  3). 

In  this  process  of  identifying  the  determinants  of  production  cost,  the  analyst  must  draw  heavily  on 
his  own  technical  skills  plus  those  of  knowledgeable  people  in  the  field.  He  must  bear  in  mind  that  since 
major  design  problems  are  presumably  resolved  before  production,  any  difficulty  in  figuring  out  how  to 
design  an  article  does  not  bear  directly  on  the  "complexity"  of  production  (which  is  more  affected  by  con- 
siderations of,  say,  machining  tolerance,  electronics  interconnections,  etc.). 

Typically  these  same  cost  determinants  are  identified  in  CKRs  for  research  and  development.  The  costs 
of  fabricating  an  R&D  Test  article  are  generally  correlated  with  the  cost  of  the  production  article.  How- 
ever, design  and  some  testing  costs  are  more  directly  related  to  factors  such  as  state-of-the-art  advance; 
in  short,  to  differences  from  the  prevailing  level  of  technology,  rather  than  to  absolute  values  of  parti- 
cular parameters.  These  points  raise  various  methodological  Issues  which  we  have  addressed  In  a number  of 
studies;  they  are  discussed  in  more  detail  in  Sec.  3. 

Costs  are  also  Influenced  by  "program"  variables,  i.e.,  by  characteristics  not  of  the  equipment  but 
of  the  management  philosophy  and  environment.  For  example,  we  have  included  variables  within  a CER  to 
account  for  schedule  considerations  (was  the  schedule  "normal"  or  "crash"?),  and  for  differences  in  parts- 
screening  and  reliability-testing.  Other  program  variables  reflect  differences  in  management  structure  and 
accounting  procedures  (Was  development  or  system  integration  done  within  a government  laboratory  or  by  a 
private  contractor?).  In  ongoing  studies  of  computer  software  we  find  these  program  variables  are  highly 
significant . 


One  restriction  In  this  process  of  identifying  cost-determining  parameters  is  that  the  parameters  be 
"available."  There  are  two  aspects  of  this  availability: 

1.  The  parameter  should  be  one  which  is  specified  early  in  the  life  cycle,  and  does  not 
require  completion  of  design  before  it  is  known.  For  example,  the  number  of  electronic 
components,  or  the  weight  and  input-power  requirement  of  electronics  systems  are  gen- 
erally not  determined  until  the  design  is  completed.  Thus  they  are  not  useful  as 
independent  parameters  in  CKRs  which  are  to  be  used  prior  to  design  completion.1 

2.  The  parameters  should  be  easily  and  unambiguously  measurable.  To  illustrate:  recent 
airborne  radar  programs  have  emphasized  advanced  signal  processing  techniques  (primarily 
for  clutter  processing  and  electronic  countermeasures).  Accounting  for  the  use  of 
particular  techniques  and  for  varying  levels  of  system  capability  has  proven  to  be 
extremely  difficult.  Moreover,  because  each  system  has  proven  to  be  virtually  unique, 

a homogeneous  data  base  cannot  be  developed. 


A 

Parameters  such  as  the  quantity  procured  and  the  level  of  inflation  are  discussed  under  Data  Norma  1 i zat ion 
(Sec.  2.5). 

For  some  applications,  such  as  spacecraft  avionics,  weight  and  power  are  initial  inputs  in  the  form  of 
design  constraints.  In  these  instances  considerable  care  must  be  exercised:  the  data  base  should  be 
restricted  to  those  systems  also  developed  under  weight  or  power  constraints. 


As  a consequence  we  have  been  forced  to  use  cruder  estimating  parameters  for  radar  receivers 
and  signal  processors  (e.g.,  $5,000  per  receiver  channel,  or  $20,000  for  a particular  type 
of  signal  processing). 

Similarly,  missile  accuracy  has  often  been  postulated  as  an  explanatory  variable  for  missile 
guidance  costs.  However,  accuracy  measures  (CEP,  etc.)  for  tactical  missiles  are  seldom 
defined  in  consistent  terms  (Is  it  single-shot  or  salvo?  Is  the  target  maneuvering?  At 
what  speed  and  with  what  type  of  signature?  How  is  distance  from  the  target  measured,  etc?). 

Where  these  criteria  cannot  be  met,  the  analyst  must  use  other  parameters. 

When  no  cause-and-ef feet  relationships  can  be  postulated,  a common  resort  is  to  settle  for  a "mere" 
correlation  with  some  measurable  parameter.  This  point  warrants  elaboration.  As  noted  previously,  the  cost 
analyst  generally  strives  to  identify  a basic  cause-and-ef feet  relationship  between  independent  and  depen- 
dent variables.  Hopefully,  statistical  procedures  in  conjunction  with  empirical  data  will  establish  that 
the  dependent  variable  is  indeed  dependent.  However,  statistical  correlations  may  actually  reflect  the  fact 
that  the  dependent  and  "independent"  variable (s)  are — in  reality — common  effects  of  some  other  (generally 
unidentified)  cause.  Schematically,  the  relationship  is  illustrated  below: 


Effects 


and  Y2  may  be  strongly  correlated  through  their  common  association  with  X.  . For  purposes  of  pre- 
dicting Y2  it  may  be  perfectly  satisfactory  to  make  use  of  this  correlation — if  Y^  can  be  readily  pre- 
dicted. However,  the  analyst  must  bear  in  mind  that  direct  analysis  of  the  relationships  between  Y^  and 
Y2  (e.g.,  what  would  be  the  effect  on  Y2  if  Y|  was  purposely  increased?)  is  generally  infeasible. 

Actual  examples  of  these  points  are  plentiful.  For  example,  estimates  of  recurring  production  cost  are 
often  used  to  predict  non-recurring  investment  cost.  In  fact,  both  costs  are  commonly  influenced  by  system 
complexity  and  size.  Electronics  weight  and  power  consumption  are  other  common  examples  of  a dependent 
parameter  being  used  as  an  independent  variable  in  cost  analysis.  Again,  if  these  variables  can  be  estab- 
lished by  some  other  means,  they  can  be  used — with  care — to  estimate  costs.  However,  direct  tradeoffs 
between,  say,  cost  and  weight  requires  a different  analytic  approach. 

2.3.2  Selecting  a Functional  Form  of  the  CER 

In  this  subtask  the  analyst  hypothesizes  a specific  mathematical  form  for  the  estimating  equation.  For 
example,  will  the  equation  be  of  a simple  linear  form  (Y  = Aq  + A^  Xj  + A2  X2  + . . .)?  Or  logarithmic  (e.g., 
Y = Aq  + A^  log  X^  + . . . , or  log  Y = Aq  + A^  log  X^  + . . .)?  Or,  is  the  underlying  relationship  between 
dependent  and  independent  variables  such  that  a complicated  mathematical  expression  should  be  used? 

Perhaps  the  most  basic  choice  is  between  linear  and  logarithmic  forms.  All  too  often  the  choice  seems 
to  be  governed  by  the  available  supply  of  graph  paper,  or  by  the  often  misleading  comparison  of  statistical 
indicators.*  In  fact,  there  is  (or  should  be)  a logic  associated  with  this  basic  selection.  The  choice 
should  be  governed  by  the  analyst's  understanding  of  how  costs  vary  with  respect  to  changes  in  the  independent 
parameters.  Consider  an  example  in  which  power  output  is  the  independent  variable.  A key  question  is:  Do 
costs  scale  proportionately  with  power  (i.e.,  an  X percent  change  in  power  leads  to  a Y percent  change 
in  cost)?  Or  do  cost  and  power  relate  in  such  a way  that  there  is  a constant  cost  per  each  unit  variation 

in  number  of  watts?  Mathematically,  the  proporat ionate  or  logarithmic  representation  is  —■  « ~ , while  the 

linear  representation  is  AY  « AX  . ^ "Semi  logarithmic"  forms  may  also  be  specified  in  which  one  or  another 
of  the  variables  is  expressed  in  logarithmic  form. 

* 

There  are  a number  of  computer  programs  which  provide  regression  statistics  for  a variety  of  candidate  mathe- 
matical expressions.  For  many  (If  not  most)  of  these  the  direct  comparison  of  the  Correlation  Coefficient  is 
misleading.  The  problem  is  that  the  residuals  are  not  expressed  in  consistent  terms  (this  point  is  discussed 
in  Ref.  13). 

+A  orollary  to  this  distinction  between  linear  and  logarithmic  expressions  is  that  the  error  terms  in  the 
statistical  "models"  which  correspond  to  these  two  types  of  equation  are  different.  The  error  in  the  linear 
function  is  assumed  constant  for  all  values  pf  the  independent  variable.  It  increases  with  larger  values  of 
this  variable  for  the  logarithmic  form  (see  Ref.  14). 

Analysts  should  also  note  that  most  curve- f it t ing  procedures  treat  the  linear  and  logarithmic  forms 
differently — the  former  minimizes  the  absolute  deviations  while  the  latter  minimizes  proportionate  deviations. 
The  consequence  of  the  difference  is  that  statistical  indicators  for  the  two  equation-forms  cannot  be  directly 
compared . 
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The  preceding  discussion  has  dwelled  upon  the  simplest  of  mathematical  relationships  between  depen- 
dent and  Independent  variables.  Often,  more  complex  equation  forms  are  required.  For  example.  In  studies 
of  electronics,  RF  frequency  is  a common  independent  variable.  In  analyzing  the  relationships  between  the 
cost  of,  say,  radar  preamplifiers  or  dish  antennas  and  frequency,  we  have  found  two  counteracting  influences. 
On  the  one  hand,  higher  frequencies  (and  correspondingly  smaller  wavelengths)  mean  smaller  physical  units, 
hence  lower  material  costs.  On  the  other  hand,  higher  frequencies  also  mean  more  stringent  control  toler- 
ances and  corresponding  higher  costs.  The  combined  effect  of  these  two  influences  is  a U-shaped  function 
as  illustrated  in  Fig.  2.2. 

Statistical  procedures  can  then  be  used  to  calibrate  and  verify  the  postulated  functional  form.  These 
are  discussed  in  Sec.  2.5,  following  a brief  discussion  of  data  collection  and  normalization. 

2.4  Data  Collection 

The  next  step  is  the  collection  of  relevant  cost  and  technical  data  from  historical  projects  which 
can  be  used  to  verify  the  postulated  CER.  If  creativity  and  technical  insight  are  hallmarks  of  CER  for- 
mulation, then  tenacity  and  stamina  are  the  watchwords  of  data  collection.  In  virtually  all  cost  analyses 
the  fundamental  limitation  is  the  dearth  of  valid,  consistently  defined  data,  and  most  of  the  analyst's 
effort  seems  to  be  required  for  this  phase  of  activity.  Figure  2.3  is  one  analyst's  view. 

To  compile  data  for  the  dependent  and  independent  variables  it  is  generally  best  to  go  where  the  data 
was  originally  generated,  i.e.,  contractor  accounting  and  engineering  records.  Care  must  be  taken  to  ensure 
a full  understanding  of  the  data — What  cost  elements  are  included?  Does  the  itemization  of  hardware  ele- 
ments correspond  to  the  Work  Breakdown  Structure?  In  what  year  were  the  costs  incurred?  Are  there  any 
government-furnished  equipments  involved?  What  was  the  schedule  of  production  items?  Etc.??  Because  of 
the  perceived  sensitivity  of  cost  information,  and  because  of  the  highly  varied  accounting  systems  in  use 
throughout  industry,  the  analyst  can  anticipate  that  a major  share  of  his  time  and  effort  will  be  given  to 
this  phase  of  study.  Hopefully  the  newly  implemented  CCDR  system  will  alleviate  these  problems  within  the 
U.S.  Department  of  Defense. 

2.5  Data  Normalization 


Prior  to  incorporating  data  within  the  postulated  estimating  relationships  the  analyst  must  ensure  that 
the  data  are  consistently  defined.  This  includes  both  technical  and  cost  information.  Normalization  of 
technical  data  includes  adjustments  to  ensure  that,  say,  the  specific  impulse  for  various  rocket  motors  is 
calculated  at  the  same  altitude,  or  that  radar  detection  range  is  normalized  to  the  same  target  cross- 
sectional  area,  number  of  pulses,  false  alarm  probability,  etc.  In  practice,  the  effort  required  to  estab- 
lish consistency  in  technical  descriptors  (independent  variables)  can  be  formidable. 

Aside  from  uniform  definitions  of  just  what  costs  are  included  within  the  recorded  expenditures,  there 
are  several  considerations  required  to  ensure  that  costs  are  displayed  in  consistent  and  comparable  terms. 
These  include  the  effects  of  (1)  year-to-year  price  level  changes,  (2)  productivity  or  manufacturing  tech- 
nology changes,  (3)  the  "tiering"  of  contractors  and  subcontractors,  and  (4)  "learning,"  or  cost-quantity 
effects  upon  production  costs. 


Price-Level  Changes.  A conventional  element  in  cost  analysis  is  the  normalization  of  cost  data  from 
different  years  to  a "constant  dollar"  basis,  thus  ensuring  comparability  among  the  data  and  the  derived 
CERs.  DoD  costing  procedures  call  for  cost  estimates  to  be  developed  in  terms  of  the  fiscal  year  following 
the  date  of  the  study.  Thus,  cost  estimates  developed  in  January  1979  must  be  presented  in  FY  1980  dollars. + 


In  the  United  States  the  Office  or  Assistant  Secretary  of  Defense  (Comptroller)  issues  price  level 
indices  for  use  in  weapons  systems  cost  analysis;  these  are  generally  presented  for  Procurement  and  for  RDTE, 
with  a single  index  covering  a broad  variety  of  equipments  and  systems.  The  military  services  issue  a some- 
what more  detailed  set  of  price-level  indices.  For  example,  the  Army  provides  separate  indices  for  the 
following  equipment  and  life-cycle  categories:^ 


• Aircraft 

• Weapons/track  combat  vehicles 

• Missiles 

• Electronics 

• Ammunition-hardware 

• OPA  (Other  Procurement,  Army) 

• RDTE  (Research,  Development,  Test  and  Engineering) 

• OMA  (Operations  and  Maintenance,  Army) 

• MPA  (Military  Pay,  Army) 


To  be  most  rigorous  the  analyst  would  limit  his  search  to  actual  recorded  costs.  However,  design  studies 
and  engineering  estimates  must  often  be  used.  Obviously,  estimates  are  not  as  desirable  as  "actuals," 
but  with  reasonable  care  to  ensure  against,  say,  undue  optimism  in  the  interest  of  being  low  bidder,  care- 
fully developed  estimates  can  be  valuable. 

+Price  level  indices  are  actually  used  for  two  purposes:  (1)  to  normalize  past  data  as  part  of  the  process 
of  developing  a CER  and  (2)  to  develop  projections  of  future  costs  to  "then-year"  information  for  future 
budgetary  planning. 

* 

See  AMCCP-ER  memo,  "Inflation  Guidance,"  dated  10  December  1975. 
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The  Air  Force  makes  forward  projections  for  Airframe,  Engine,  and  Avionics  (for  both  Development  and  Pro- 
duction). 

Associated  with  the  use  of  these  indices  are  several  considerations  which  represent  significant  po- 
tential errors  in  cost  analysis: 

1.  The  additional  detail  for  equipment  categories  and  the  periodic  updating  of  price- level 
indices  are  largely  restricted  to  future  years.  That  is,  they  are  addressed  to  the  second 
objective  of  price-level  normalization  noted  previously — projection  of  future  costs  to 
"then-year”  information. 

However,  the  first  requirement  of  the  cost  analyst  is  an  index  to  adjust  costs  from 
prior  years  to  the  base-line  year  of  his  estimates.  indices  for  prior  years  are  not  regularly 
updated,  nor  are  revisions  at  finer  levels  of  equipment  detail  directly  issued. 

2.  Even  with  the  most  detailed  indices  there  still  is  considerable  variation  among  the 
constituent  hardware  elements  (e.g.,  missile  tail  fins,  fuel  pumps,  combustion  chambers...) 
in  terms  of  types  of  material  and  labor,  and  the  relative  mix  of  these  two  major  inputs  to 
the  development  and  production  process.  In  recent  years,  which  have  been  characterized  by 
sharp  and  varied  price-level  increases,  the  'averaging"  within  equipment  catogories  can 
mask  considerable  variability  in  actual  price-level  effects. 


3.  In  practice,  the  available  price  level  indices  (which  are  based  on  labor  and  material 
inputs)  are  used  to  adjust  the  costs  which  have  been  recorded  for  particular  subsystem  and 
components.  Thus,  for  example,  the  cost  of  a missile  subsystem,  which  was  incurred  in — say — 
1973,  is  adjusted  to  1977  dollars  by  multiplying  the  cost  by  a 1973-to-1977  index.  But  the 
analyst  should  note  that  the  recorded  cost  of  the  missile  subsystem  is  an  output  measure. 

It  reflects  the  cost  of  individual  inputs  (labor,  material...),  together  with  the  process 


by  which  inputs  are  combined,  to  yield  the  desired  subsystem  or  component  performance.  If 

there  is  year-to-year  variation  in  this  process,  another  source  of  error  may  be  introduced. 

This  is  amplified  in  the  following  discussion  and  in  Sec.  3. 

Productivity  Changes.  The  price  level  indices  discussed  in  the  preceding  paragraphs  are  used  basically 
to  account  for  changes  in  the  unit  prices  of  the  inputs  to  R&D  and  production,  i.e.,  in  the  prices  of 
labor,  material,  equipment,  etc.  A related  consideration  is  the  effect  of  changes  in  manufacturing  pro- 
ductivity, i.e.,  in  the  cost  per  unit  output  stemming  from  changes  in  the  technology  used  in  producing 
equipment.  The  most  striking  current  example  is  probably  in  small  computers  and  calculators,  where  the 
cost  of  a given  level  of  computational  capability  has  been  dropping  dramatically  as  new  circuit  tech- 
nologies are  implemented.  These  year-to-year  changes  in  the  cost  per  unic  output  are  not  captured  by 
price  level  indices. 

For  many  types  of  equipments  these  year-to-year  changes  in  productivity  are  not  pronounced.  But,  for 
some — notably  avionics — they  are,  and  data  from  programs  more  than  a few  years  old  should  be  used  only  with 
caution.  Most  cost  analyses  do  not  take  these  effects  into  account.  Some  methodological  work  to  incor- 
porate these  changes  is  discussed  in  Sec.  3. 

Contractor  Tiering.  Normalization  of  contractor  tiering  is  another  requirement  to  ensure  that  cost 
figures  from  different  sources  are  comparable  and  consistently  defined.  In  complex  weapon  systems,  in- 
dividual components  of  hardware  are  often  marked  up  with  G&A  and  profit  through  successive  tiers  of  sub- 
contractors and  a final  weapon  system  contractor.  For  example,  small  component  suppliers  (e.g.,  mirror 
manufacturers)  sell  their  products  to  a subystem  manufacturer  (e.g.,  an  optics  subsystem  supplier),  who 
in  turn  sells  the  subsystem  to  a weapon  system  contractor.  The  prices  quoted  for  a given  component  may 
differ  markedly  depending  upon  which  tier  is  reporting. 

One  set  of  representative  factors  which  account  for  the  price  differentials  are  illustrated  in 
Table  2.2  (these  factors  are  based  upon  studies  of  US  Army  systems,  primarily  SAFEGUARD).  In  studies  of 
spacecraft  avionics  we  have  observed  a different  set  of  factors  for  NASA-sponsored  programs,  with  an  approx- 
imately 25%  tiering  factor  at  each  stage. 

"Learning,"  or  Cost  Quantity  Relationships.  A standard  tool  in  analyzing  the  recurring  costs  of  pro- 
duction items  is  the  "learning  curve."  There  is  an  extensive  literature  dealing  with  the  principles  and 
applications  of  this  concept;  the  reader  can  find  a complete  discussion  in  Refs.  15  and  16. 

In  brief,  the  learning  curve  is  a means  of  normalizing  the  effects  of  different  total  quantities  upon 
the  cost-per-unit . As  originally  conceived,  the  learning  curve  reflected  the  increased  efficiency  of  labor 
(represented  in  terms  of  labor-hours  per  unit  output)  as  more  and  more  units  are  produced.  In  practice, 
learning  curves  are  applied  to  the  total  recurring  cost  of  production  items.  The  effect  of  "learning"  is 
portrayed  as  a decrease  in  (1)  the  cumulative  average  cost  per  unit  as  a function  of  quantity  produced,  or 
(2)  the  cost  per  unit  of  the  most  recent  unit  as  a function  of  quantity.  We  have  generally  expressed  the 
learning,  or  cost-quantity,  relationship  in  terms  of  cumulative  average  cost  per  unit.  The  mathematical 
expression  has  the  exponential  form: 


See  Aeronautical  Economic  Escalation  Indices,  Comptroller,  Aeronautical  Systems  Division,  duly  1975. 
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where  A ■ first  unit  cost 

X * quantity 

* 

b * an  exponent,  0 < b < 1 , determined  by  the  extent  of  learning 


Learning  curves  are  used  in  two  ways:  (1)  to  normalize  historical  data  so  that  variations  in  recorded 
costs  due  to  differences  in  the  quantity  produced  may  be  properly  taken  into  account,  and  (2)  to  account  for 
the  effects  of  quantity-produced  in  equations  set  forth  to  estimate  future  costs. 

The  analysis  of  historical  cost-quantity  data  is  complicated  by  several  factors.  Costs  for  various 
quantities  of  output  necessarily  are  recorded  over  a period  of  time,  during  which  a number  of  other  factors 
may  influence  unit  cost.  Notable  among  these  are: 

• Changes  in  overhead.  General  and  Administrative  expenses,  and  profit  which  may  change 
in  some  systematic  way  over  time  and  thus  influence  the  unit  cost. 

• Changes  in  production  rate.  Most  of  the  detailed  industrial-engineering  analyses  we 
have  seen  have  emphasized  the  importance  of  production  rate  in  determining  production 
cost.  And  yet,  typically,  production  rate  is  not  explicitly  considered  in  cost  analyses. 

If  the  rate  changes  markedly  during  an  overall  production  run,  there  may  be  substantial 
effects  upon  unit  production  cost. 

• Various  modif ications  and  engineering  changes  in  the  end  product  which  often  take  place 
throughout  a production  run. 

• Different  contractual  terms  which  may  influence  the  recorded  costs  and/or  actual  effi- 
ciencies in  the  production  process.  One  can  readily  imagine  the  influences  from  a 
shift  to,  say,  a fixed-price-incentive  contract  from  a cost-plus-fixed-fee  contract. 

In  these  circumstances,  the  analyst  can  only  strive  to  identify  these  influences  and  adjust  the  data  for, 
say,  measured  increases  in  overhead  or  cost  due  to  modifications. 


Even  with  careful  accounting  of  these  influences,  there  is  considerable  room  for  error  in  specifying 
actual  values  for  learning  curves  (the  learning  rate  for  a given  class  of  equipment  may  actually  differ 
from  contractor  to  contractor,  and  among  different  models  of  the  same  class).  Thus  it  is  prudent  to  select 
a "standard"  quantity  (to  be  used  in  comparing  unit  costs)  that  is  actually  within  the  actual  range  of 
available  data.  For  example,  if  tactical  missiles  are  being  compared,  and  the  range  of  actual  quantities 
is  between  100  and  1,000  units,  then  comparisons  should  be  made  for  a standard  quantity  of,  say,  500.  Pro- 


jection of  data  to  a first-unit  cost  increases  the  possibility  of  error  in  the  comparison. 
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Ca lib ra ting  and  Verifying  the  CER 

The  preceding  steps  have  identified  the  independent  variables,  postulated  a specific  mathematical  form, 
and  collected  and  normalized  relevant  data.  The  next  task  is  to  calibrate  and  verify  the  postulated  CER 
using  techniques  of  mathematical  statistics.  The  procedure  of  least-squares  curve-fitting,  or  multiple 
regression  analysis,  provide  a rigorous  and  consistent  means  of  determining  a specific  mathematical  rela- 
tionship. A full  exposition  of  the  underlying  theory  and  principles  of  mathematical  statistics  and  statis- 
tical inference  is  beyond  the  intended  scope  of  this  paper.  Suffice  it  to  say  there  are  many  texts  which 
cover  these  topics  (see,  e.g..  Refs.  10-12),  and  the  analyst  should  understand  the  assumptions  and  limita- 
tions of  this  type  of  analysis.  In  practice,  the  rigorous  requirements  concerning  the  quantity  and  charac- 
teristics of  the  data  are  seldom  fulfilled.  However,  the  deviations  and  exceptions  do  not  preclude  the  ad- 
vantageous use  of  statistical  procedures. 


The  results  of  a conventional  regression  analysis  are  (1)  coefficients  and  exponents  for  the  postu- 
lated mathematical  form,  and  (2)  a variety  of  statistical  indicators  which  describe  the  relationship  be- 
tween the  resultant  mathematical  relationship  and  the  data  in  deriving  it.  Among  the  more  useful  indicators 
are  the  following  (some  additional  discussion  of  these  indicators  is  given  in  Appendix  A): 

2 

• The  Correlation  Coefficient  (R  ).  The  percent  of  variance  in  the  dependent  variable 
explained  by  the  estimating  function. 

• The  Standard  Error  of  the  Estimate.  The  standard  deviation  of  the  dependent  variable 
from  the  computed  values. 

• t-statistics . For  each  coefficient,  the  ratio  of  the  coefficient  to  its  standard  error. 

• F- value.  The  ratio  of  the  explained  to  unexplained  variance,  a measure  of  the  signi- 
ficance of  . 


The  common  measure  of  learning  is  expressed  as  the  percent  change  in  cost  when  the  quantity  is  doubled. 

The  value  of  b for  a 90%  learning  curve  is  -0.152.  To  determine  total  cost,  the  unit  cost  expression  is 

multiplied  by  the  quantity.  Thus,  total  cost  = AX^  ^ , It  should  be  noted  the  "first  unit  cost"  is  a 
recurring  cost;  it  is  not  the  total  cost  of  producing  one  item  (which  would  include  the  costs  of  manufac- 
turing engineering,  tools,  special  test  equipment,  and  other  "non-recurring"  items). 


M I 
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Hie  correlation  coefficient  is  a good  general  indicator,  although  It  can  be  misleadingly  high.  Hie 
standard  error  of  the  estimate  is  an  important  indicator,  although  the  proport ionate  (or  percent)  deviations 
are  probably  more  illuminating.  In  fact,  the  analyst  should  examine  the  pattern  of  residuals 

(Y  , - Y . ) for  any  systematic  patterns  (more  on  this  subsequently), 

actual  estimate  7 r 

The  t-statistlcs  for  each  coefficient  should  be  high  (>1)  in  order  to  have  any  confidence  in  the  cal- 
culated values  of  the  coefficient. 

There  are  numerous  ways  that  these  indicators  can  be  deceptive  (see,  e.g.,  the  discussion  in  Ref.  11). 
As  a consequence,  statistical  indicators  must  be  viewed  with  some  caution.  However,  by  ensuring  that  the 
data  are  reasonably  well-distributed  over  the  data  space  and  that  mul t i-coll ineari ty  and  other  potential 
statistical  problems  are  reasonably  well  controlled,  the  analyst  will  find  that  these  indicators  can  be  a 
powerful  tool. 

With  all  these  mathematical  procedures,  the  ultimate  criterion  is  the  reasonableness  of  derived  re- 
lationship. Do  the  relat ionships  make  sense  in  light  of  the  analyst's  knowledge  about  the  system  and  about 
the  data  used  in  deriving  the  CER? 

What  to  do  If  the  CER  Is  no  Good 

All  too  often  the  analyst  will  find  that — at  least  for  the  first  try — he  has  not  developed  a defensi- 
ble estimating  relationship.  There  are  a number  of  potential  explanations,  and  there  are  several  recourses. 
Among  the  reasons  why  the  estimating  relationship  (CER)  is  unsatisfactory  may  be: 

1.  The  analyst  has  failed  to  identify  the  actual  cost-determining  variables  and  the  general 
nature  of  how  these  variables  influence  cost. 

2.  The  data  may  be  inaccurate  or  inadequate. 

There  are  several  things  the  analyst  can  do  in  order  to  address  these  problems: 

1.  Rcc valuate  the  identified  -independent  vaiiabl es 

The  obvious  (and  most  fundamental)  recourse  is  to  again  think  through  the  basic  process 
under  study  (such  as  the  production  of  a radar).  Are  there  other  factors  which  explain  the 
costs  of  this  process?  As  an  aid  in  identifying  these  variables  it  is  often  useful  to  examine 
the  pattern  of  residuals  from  the  "unsuccessful"  CER.  To  illustrate:  the  analyst  can  plot  a 
graph  showing  actual  cost  versus  estimated  costs  using  the  CER. 


ESTIMATED  COST 

I 


ACTUAL  COST 


Upon  examination  of  those  programs  whose  data  lie  consistently  to  one  side  of  the  45°  line  (along 
which  estimates  - actuals),  the  analyst  may  find  some  single  factor  which  stratifies  the  data  in 
this  same  pattern.  For  example,  in  previous  studies  we  have  found  stratifications  based  upon 
application  area  (airborne  versus  ground-based  usage1*),  or  differing  levels  of  environmental  or 
EMI  specification,  or  a different  design  technology  used  in  the  design  (e.g.,  with  or  without 
pulse  frequency  agility).  In  these  instances  the  CER  can  be  improved  by  addition  of  an  appro- 
priate dummy  variable  to  stratify  the  data. 

2.  Reevaluate  the  hypothesized  mathematical  fo\m  of  the  CER 

An  originally  hypothesized  linear  or  log-linear  mathematical  form  may  not  properly  reflect 
the  actual  relationship  between  independent  and  dependent  variables.  Again  an  evaluation  of  the 
residuals  from  the  "unsuccessful"  CER  may  indicate  a systematic  pattern  which  is  indicative  of 
some  heretofore-unrecognized  relationship.  For  example,  a systematic  pattern  of  "actuals"  ex- 
ceeding estimates  at  the  upper  range  of  values  may  suggest  that  some  upper  bound  of  achievable 
performance  is  being  approached.  One  might  expect  a functional  relationship  between  performance 
and  cost  to  be  of  the  following  form  (which,  in  fact,  we  have  observed): 


* 

There  are  a number  of  statistical  problems  which  are  commonly  encountered;  perhaps  the  most  notable  diffi- 
culty is  multicolllnearlty  (see  Refs.  17,  18)  which  is  generally  manifested  in  low  t-statistics  although 
the  correlation  coefficient  may  be  fairly  high. 

*I.e.,  cost  derived  by  using  actual  values  of  the  independent  variables  from  programs  in  the  data  base  as 
Inputs  to  the  estimating  function. 

^Different  packaging  and  environmental  specifications  invariably  lead  to  marked  cost  differences  between 
these  two  operating  regimes. 
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3.  attempt  to  tie vo Cop  a CCR  at  a di Relent  CeveC  ofl  detail 

This  procedure  can  be  done  either  way,  i.e.,  the  analyst  may  seek  relationships  at  a finer 
level  of  detail,  or  he  may  have  to  resort  to  a more  aggregated  relationship.  As  noted  previously 
a finer  level  of  detail  may  permit  an  expansion  of  the  data  base.  At  a finer  level  of 
detail  there  may  be  a broader  selection  of  valid  analogs.  Thus  there  may  be  a larger  sample  of 
data  with  which  to  draw  inferences  about  costs. 

When  additional  data  and/or  analogs  cannot  be  identified  at  a finer  level  of  detail,  the 
cost  analyst  may  be  forced  to  develop  the  CER  at  a higher  level  of  aggregation,  and  to  accept 
a less  defensible  estimating  relationship.  For  example,  in  one  study  of  spacecraft  Stabilization 
and  Control  (S&C)  subsystems,  we  found  that  each  of  the  S&C  subsystems  for  which  we  could  obtain 
data  was  virtually  unique.  Each  had  a peculiar  set  of  individual  end-items  (as  indicated  in 
Table  2.3),  and  we  were  unable  to  find  other  data  which  would  permit  development  of  CERs  for 
each  individual  end- item.  Consequently  we  were  forced  to  develop  an  aggregated  CER  for  the 
entire  S&C  subsystem — one  which  admittedly  did  not  account  for  the  distinct  stabilization  and 
control  components. 

4.  Settle  &ca  cciA.elaticn  until  anotheA  callable  which  can  be  estimated 

As  noted  in  an  earlier  discussion  it  is  not  always  possible  to  establish 
a causal  inference  to  explain  variations  in  cost,  and  the  analyst  mav  have  to  settle  for  a 
simple  correlation  with  another  variable.  One  common  example  is  the  estimation  of  R6D  costs 
for  high-technology  equipment  as  a function  of  the  equipment  production  cost.  Both  are 
common  effects  of  various  facets  of  size  and  complexity.  If  a reasonably  good  CER  is  avail- 
able for  production  cost,  it  can  be  used  to  estimate  the  value  of  the  independent  variable 
in  the  R&D  CER. 

Another  example  is  the  use  of  weight  or  input-power  as  independent  variables  in  many 
electronics  CERs. 

5.  "Waif  until  next  ijeai" 

Finally,  there  will  be  occasions  when  no  parametric  estimating  relat ionship  can  be  fovind. 

Some  elements  of  cost  cannot  be  linked  to  any  reasonably  sized  set  of  technical  or  program 
characteristics.  One  example  was  identified  in  a GRC  study  of  "indirect"  missile  production 
costs  such  as  manufactur ing  engineering,  engineering  changes,  and  documentat ion . Another 
category  is  Product  Improvement , for  which  we  were  unable  to  establish  any  defensible  CER 
(see,  for  example,  the  scatter  of  data  points  in  Fig.  2.4).  Upon  further  inquiry  we  found 
that  Product  Improvement  expenditures  were  largely  governed  bv  the  availability  of  "year-end" 
funds,  i.e.,  they  are  subject  to  random  quirks  of  the  budget  and  not  to  the  technical  char- 
acteristics of  the  program  itself.  In  short,  there  is  no  parametric  relationship  with  respect 
to  cost.  The  best  we  could  do  in  this  instance  was  to  suggest  an  upper  bound  of  a Product 
Improvement  cost. 

3.  EXTENSIONS  AND  REFINEMENTS 

3.1  I n t rod  uc t i on 

The  preceding  discussions  have  outlined  the  principles  and  procedures  of  parametric  cost  analysis. 

In  this  section  various  extensions  and  methodologica 1 refinements  are  discussed.  For  the  most  part  they 
are  extensions  in  the  sense  that  thev  still  follow  the  basic  precepts  of  parametric  cost  analysis;  thev 
represent  Improved  hypotheses  and  models  of  the  process  under  studv.  They  both  address  an  issue  which 
character izes  a significant  share  of  the  new  equipment  ir.d  systems  which  are  acquired  each  year  bv  private 
industry  and  the  government,  viz.,  the  pervasive  effects  of  technological  change.  Cost  analysts  have  long 
incorporated  the  effects  of  major  "jumps"  in  technology  bv  developing  separate  CERs  for,  say,  conventional 
tube  vs.  solid-state  electronics,  aluminum  vs.  titanium  aircraft  structures,  metal  vs.  f i lament -wound 
rocket-motor  cases,  etc.  However,  cost  analysts  have  generally  not  taken  into  account  the  continuous  and 
fundamental  wav  that  techno  logics  1 change  affects  manv  areas  of  government  and  industry. 

In  research  and  development,  technological  change  is  a basic  goal.  In  production,  technological 
change  direct lv  affects  the  process  of  transforming  inputs  (resources)  into  outputs  (delivered  performance) . 
Anv  analytic  technique  which  purports  to  capture  the  basic  elements  which  determine  resource  expenditures 
must  include  these  effects.  The  "extensions"  discussed  in  this  section  are  exploratory  and  obviously  in- 
complete. No  doubt,  thev  can  be  greatly  refined  and  improved.  Included  are  discussions  of 


Procedures  for  measuring  and  incorporating  state-of-the-art  advance  as  a parameter  in 
R&D  cost-estimating  relationships 
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• Procedures  for  incorporating  effects  of  technological  change  in  cost-estimating  relation- 
ships for  production  systems. 

3.2  Technological  Change  and  R&D  Programs 

3.2.1  State-of- the-Art  "Surfaces" 

There  are  numerous  studies  focussing  upon  improved  methods  of  estimating  R&D  costs  which  have  identi- 
fied state-of-the-art  advance  as  a key  cost  determinant.  Most  of  these  studies  have  been  based  upon  sub- 
jective assessments  or  rankings  of  this  advance,  or  of  some  related  consideration  such  as  "complexity" 

(see,  e.g..  Refs.  19,  20).* 

In  a study  conducted  several  years  ago  for  the  Assistant  Secretary  of  Defense,  we  developed  a quan- 
titative approach  to  measuring  advances  in  the  state-of-the-art  (SOA)  based  upon  SOA  "surfaces"  for 
specific  points  in  time.  Our  approach  stems  from  several  basic  observations: 

• The  "state-of-the-art"  (SOA)  as  a specific  level  of  performance  changes  over  time  as  new 
accomplishments  are  implemented  and  become  part  of  the  industry's  general  technical 
capabi 1 it les . 

• At  any  particular  point  in  time,  the  SOA  for  a particular  system  or  application  is  a function 
of  numerous  parameters.  For  example,  power  output  is  one  indicator  of  the  SOA  for  a radar 
transmitter,  but  specification  of  a particular  output  in  watts  requires  specific  values  or 
constraints  upon  other  parameters  such  as  frequency,  weight,  etc. 

• Subsequent  systems  with  greater  capabilities  advance  the  SOA,  the  advance  being  measurable 
in  relation  to  the  SOA  prevailing  at  the  time  the  design  approach  was  selected. 

Briefly,  our  procedure  is  to  identify  several  variables  which — to  a reasonable  degree — bound  the  state-of- 
the-art  (SOA)  in  design  achievement  for  the  subsystem  or  equipment  under  study.  These  variables  are  gen- 
erally performance  and/or  physical  descriptors,  and  must  be  chosen  so  that  they  reflect  actual  design  goals 
or  constraints.  A specific  time  period  (e.g.,  late  sixties)  is  then  chosen,  and  all  of  the  equipments 
developed  in  the  period  are  cataloged  in  terms  of  the  chosen  technology  variables.  A "surface"  is  then 
fitted  to  the  data  for  actual  equipment,  the  surface  being  defined  within  the  space  described  by  the  tech- 
nology parameters. 

To  illustrate,  in  a study  of  avionics  computers  three  technology  variables  were  set  forth  as  being 
representative  of  the  computer  state-of-the-art.  A surface  was  fit  to  six  data  points  representing  six 
computers  developed  in  the  1967-1968  time  period,  the  results  being  illustrated  in  Fig.  3.1.*  The  advance 
implicit  in  any  computer  developed  in  the  ensuing  time  period  is  then  measurable.  In  effect  this  procedure 
is  a quantitative,  objective  alternative  to  subjective  qualitative  assessments  of  "complexity"  or  "technol- 
ogical advance  rating. "5 

While  this  approach  is  felt  to  be  a valid  and  useful  representation  of  technological  change,  it  does 
have  two  fundamental  limitations.  First,  in  order  to  "fit"  SOA  surfaces  for  successive  time  frames,  a con- 
siderable sample  of  data  (i.e.,  number  of  different  development  items)  is  required.  For  some  items,  such 
as  commercial  computers,  where  there  have  been  many  development  projects,  this  is  no  great  problem.  For 
most  space  and  military  systems,  as  well  as  other  commercial  equipment,  there  have  been  fewer  development 
items  which  can  be  included  in  any  one  reasonably  homogeneous  set.  Thus,  there  is  insufficient  data  to 
specify  a surface  for  each  period  of  time.  This  drawback  limits  the  usefulness  of  the  approach  for  R&D 
costing  of  many  high-technology  systems. 

A second  limitation  is  that  the  time  period  for  each  surface  is  arbitrarily  specified;  thus  the  pro- 
cedure lacks  rigor  in  the  sense  that  another  analyst  might  specify  a different  time  period — and  will,  in 
general,  obtain  different  results.  In  short,  two  analysts  working  independently  (and  using  the  same  data 
and  technical  parameters)  may  not  come  to  the  same  analytic  conclusions.  As  a result,  our  recent  studies 
have  focussed  on  variants  of  this  overall  approach  which  avoid  these  particular  shortcomings. 

3.2.2  Variants  of  the  "Surface"  Approach 

There  are  several  variants  of  this  overall  approach  which  permit  economies  in  the  size  of  the  data 
sample,  and  avoid  the  arbitrary  stipulation  of  time  periods  . . . all  the  while  preserving  the  basic  con- 
cept. One  approach  we  have  explored  is  to  develop  a more  generalized  equation  for  the  surface  in  which 

— _ 

Still  other  studies  have  been  concerned  with  measuring  technological  advance  for  purposes  of  technological 
forecasting  (e.g..  Ref.  21). 

^The  rationale  and  "mechanics"  of  this  procedure  are  given  in  Ref.  22.  An  ellipsoidal  form  was  used  to 
account  for  the  "diminishing  returns"  as  increased  values  of  any  particular  performance  parameter  are 
sought. 


This  work,  and  its  initial  application  to  tactical  missile  propulsion  and  guidance  has  been  fully  documented 
in  contract  reports  (Ref.  23)  and  in  the  open  literature  (Ref.  22,  24). 
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the*  equation's  parameters  are  expressed  as  a function  of  time  (instead  of  fitting  a specific  surface  to 
data  for  each  individual  time  period).  In  the  course  of  this  work  we  identified  related  approaches  which 
proved  simpler  to  use.  For  example,  Barr  and  Knight  examined  various  forms  of  economic  "productions 
functions"  in  an  attempt  to  identify  relationships  which  account  for  technological  advance."  Among  their 
investigations  was  a multiple  regression  analysis  Including  various  technical  characteristics  and  calendar 
time.  Alexander  and  Nelson  of  RAND*-^  have  set  forth  a related  approach  which  we  have  implemented  as  follows 


• A limited  number  of  technical  characteristics,  which  (reasonably)  encompass  the  technology 
of  a particular  class  of  subsystems,  are  set  forth  as  independent  variables  in  a multiple 
regression  equation. 

• The  dependent  variable  is  some  consistently  defined  calendar  milestone  for  each  subsystem 
in  the  sample  (such  as  completion  of  qualifies  ion  test). 

• The  result  of  a multiple  regression  exercise  is  a function  which  can  be  interpreted  as 
an  expected  date  Yg  for  achievement  of  a given  set  of  technical  characteristics  (which 
have  been  specified  as  independent  variables).  Y is  shown  graphically  in  Fig.  3.2. 

• The  residuals  Y - Y . can  be  used  as  indicators  of  the  extent  to  which  each  sub- 

e actual  + 

system  was  "before  its  time,"  or  may  have  lagged  behind  the  average.  This  is,  in  effect, 
a measure  of  relative  technological  advance. 


An  KiusCuiticu  the  "Tone  A^picach . 
approach  focussed  on  general  purpose  avionics  computers, 
set  forth  as  indicators  of  technological  capability,  or 


One  study  which  investigated  the  time  regression 
The  following  technical  character 1st ics  were 
state-of-the-art : 


1.  The  speed  of  the  computer  in  operations  per  second 

2.  The  density  of  the  central  processing  unit  in  pounds  per  cubic  foot 

3.  The  number  of  distinctive  instruction  types  in  the  computer  repertoire  (add,  shift  left,  etc.). 

Technical  data  and  the  date  of  development  completion  were  compiled  for  twenty-six  airborne  and  space  com- 
puters. They  were  then  vised  In  a multiple  regression  analysis  to  establish  an  equation  which  can  be  inter- 
preted as  the  estimated,  or  expected,  date  of  development-completion  as  a function  of  the  three  technology 
parameters.  The  resulting  equation  is 


Yp  * 1961 . 3 + 0 . 0 3X j + 0.015Xo  + O.ObX^ 


(3.1) 


where 


Yp  * Expected  year  of  development  completion 
Xj  a Number  of  distinct  instruction  types 
X , * Number  of  operations  (thousands)  per  second 
Xj  - CPU  densfty  (lb/ft3) 


The  next  step  was  to  examine  the  residuals,  or  deviations,  of  the  actual  data  from  the  estimating  function. 
These  are  illustrated  in  Fig.  3.3.  The  26  computers  are  shown  as  they  lie  above  and  below  the  45°  line. 

A point  above  the  line  corresponds  to  an  actual  date  of  completion  earlier  than  expected — in  short , the 
computer  was  "ahead  of  its  time."  Conversely,  computers  falling  below  the  line  were  "behind  their  time", 
i.e.,  on  the  basis  of  their  technical  performance  capability  they  would  have  been  expected  to  be  through 
development  at  an  earlier  date.  Thus,  the  residual,  (Yp  - Y^,^^),  is  interpretable  as  an  indicator  of 
state-of-the-art  advance. 


It  is  our  hypothesis  that  extra  development  effort  is  required  for  a computer  to  be  ahead  of  its 
time,  and  that  being  "behind  the  times"  should  be  correlated  with  a reduced  effort,  l.e.,  it  Is  hypothesized 
that  the  residual  (or  SOA  advance)  is  positively  correlated  with  development  cost. 


As  shown  in  Fig. 
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the  general  equation  for  an 
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is 


where  the  a ^ , a,,  «ij  ...  are  the  zero  intercepts  along  the  respective  axes.  We  are  examining  a generalized 
form  in  which  these  intercepts  are  expressed  in  linear  functions  of  time. 


Wo  are  imputing  more  to  these  residuals  than  is  conveniently  done  in  regression  analysis.  The  usual 
assumption  is  that  the  dependent  variable  is  represented  by  the  estimating  function,  plus  an  error  term 
which  is  represented  by  the  pattern  of  residuals.  In  this  particular  approach  it  is  assumed  that  there 
is  information  remaining  in  the  pattern  of  residuals,  in  addition  to  the  still-remaining  error  function. 
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This  leads  to  the  second  step  in  this  overall  cost-estimating  procedure,  viz.,  incorporation  of  SOA 
advance  in  a cost-estimating  relationship.  The  computers  in  the  sample  include  both  airborne  and  space 
computers.  Typically,  these  two  categories  are  fundamentally  different  in  environmental  specifications, 
reliability  and  test  requirements,  packaging,  and  other  considerations.  Thus  the  sample  should  be  strati- 
f led . 

The  computer  data  base  also  includes  another  basic  division:  an  older  group  of  "hard-wired,"  syn- 
chronous computers,  and  a more  recent  set  of  synchronous,  microprogrammable  machines.  This  distinction  is 
also  hypothesized  to  have  a significant  impact  upon  cost. 

These  two  classifications  (represented  as  dichotomous,  or  dummy  variables)  were  included  with  SOA 
advance  in  an  estimating  equation,  v'ith  the  following  results: 


Y = 6.11  + 2.7  Xx  - 4.7  X2  + 14.8  X3  (3.2) 

where 

Y * R&D  cost  (millions  of  1974  dollars) 

X^  = ASOA  (years) 

^ s 1»  for  microprogrammable  computers 

2 0,  for  synchronous  computers 

^ 1,  for  space  computers 

3 C,  for  airborne  computers 

To  use  these  procedures  in  evaluating  a proposed  new  computer  R&D  effort,  the  analyst  would  observe 
the  following  steps: 

• The  individual  technical  characteristics  of  the  proposed  computer  are  inserted  as  X..  , X~  , 

and  X~  of  Eq.  3.1.  Y is  calculated. 

3 e 

• The  proposed  date  of  R&D  completion  is  used  as  ' 

• Yg  - YACTUAL  *S  ca^cu^ate(^»  an£*  becomes  in  Eq . 3.2. 

• Values  of  X^  and  X^  in  Eq.  3.2  are  also  specified  for  the  proposed  program. 

• Using  Eq.  3.2,  calculate  Y , the  R&D  cost. 

Refioiementi  ofa  R&S ouA.CH- Estimating  Equation*) . The  simple  linear  relationship  between  R&D  cost  and 
state-of-the-art  advance  given  in  Eq.  3.2  is  not  altogether  satisfactory,  and  in  recent  studies  we  have 
been  refining  it.  First,  there  are  various  cost  elements  associated  with  R&D  which  are  independent  of  SOA 
advance;  thus  a lower,  or  "fixed,"  cost  will  be  approached  as  SOA  advance  diminishes.  Similarly,  there  is 
an  upper  bound  on  achievable  SOA-advance  even  though  funds  might  be  unlimited.  Thus,  one  would  expect  a 
cost  relationship  asymptotic  to  some  upper  limit  of  SOA  advance.  In  short,  we  hypothesize  a complete 
functional  form  as  follows: 


We  have  examined  several  subsystems  for  which  the  span  of  available  data  did  permit  testing  of  this 
hypothesis.  For  example,  cost  estimating  relationships  have  been  developed  in  this  form  for  communications 
multiplexers,  traveling-wave  tube  amplifiers,  and  solid  rocket  motors. 
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» 0.7,  F-statistic  * 8.6  (17  data  points),  t-statistics  are  4.3,  1.4,  and  3.3  respectively. 
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T hi*  V\!  ( til  Pc  *tQc’  \nkinc  C Alt*  til  cd . A complementary  approach  to  measuring  state-of-the-art  advance  is 
called  "Delta  Performs  nee. T'  While  quite  similar  in  principle  to  the  time  regression  approach,  the  pro- 
cedures of  use  differ  in  two  respects: 

• A separate  relationship  with  time  is  developed  for  each  individual  technical  parameter. 

• Rather  than  using  the  entire  sample  of  historical  programs  to  establish  an  ave rage  level 

of  technology  as  a function  of  time,  the  baseline  level  of  technology  (from  which  residuals 
are  measured)  is  established  from  those  programs  in  the  sample  which  represent  the  most 
advanced  capability. 

A summarv  of  this  approach  is  given  in  Fig.  3.4. 

The  analysis  begins  with  the  specification  of  a summary  set  of  performance  or  design  characteristics 
which  are  considered  indicative  of  the  state  of  technology  for  the  subsystem  under  study.  A prerequisite 
for  using  the  delta  performance  technique  is  that  these  characteristics  be  independent  of  one  another  in  the 
sense  that  specifying  a particular  value  of  one  characteristic  does  not  require  specifying  values  for  the 
others.  To  illustrate,  transmitter  power  per  unit  weight  and  receiver  sensitivity  are  primary  technology 
descriptors  for  transceivers;  yet  they  are  unrelated  to  one  another  in  the  sense  that  one  does  not  act  as 
a constraint  on  the  other. 

The  available  technical  data  are  then  examined  to  identify  the  "dominant"  items  over  time  (as  illus- 
trated in  Fig.  3.4),  and  the  particular  development  items  which  have  the  most  advanced  capability  at  each 
point  in  t Imt*  are  selected.  These  are  used  to  develop  a single-variable  expression  as  a function  of  time. 

For  example,  from  a sample  of  13  spacecraft  transceivers,  three  were  found  to  be  dominant  in  terms 
of  receiver  sensitivity  (Mariner  IV  achieved  a sensitivity  of  -150  dBm  in  1964;  Mariner  '69  set  a new 
standard  of  -153  dBm  in  1968,  while  the  M- series  transceiver  achieved  -154  dBm  in  1975). 

The  dominant  programs  are  then  used  in  developing  a regression  relationship  between  the  technology 
parameter  and  time.  In  the  example  of  the  spacecraft  transceivers,  the  following  expression  was  derived. 

Xpi  = 149  + 0.339  (Y  - 1960)  (3.3) 

where  * receiver  sensitivity,  -dBm 

Y » calendar  year 

In  effect,  Xp^  is  an  expected  level  of  technology  (in  terms  of  receiver  sensitivity)  as  a function  of 
calendar  time."  Actual  program  values  are  measured  as  they  fall  above  or  below  Xjrj  at  any  particular  point 
in  time.  We  hypothesize  that  the  residuals  ~ Xgj),  which  represent  indicators  of  state-of-the-art 

advance,  are  correlated  with  DDTE  cost. 

Other  technology  parameters  are  analyzed  in  a similar  manner.  The  residuals  from  these  analyses  are 
then  used  as  separate  terms  in  a DDTE  cost-estimating  function. 

Some  Evaluative  Remarks 

There  are  a number  of  features  common  to  each  of  these  approaches,  and  each  offers  some  distinctive 
characteristics.  Each  is  based  upon  the  assumption  that  technology  can  be  represented  in  terms  of  physical 
or  performance  variables  which  are  readily  quantified.  This  is  not  always  the  case;  for  example,  main- 
special  signal  processing  capabilities  cannot  be  readily  quantified,  nor  can  special  thrust  control  features 
of  rocket  engines. 

There  are  some  distinct  differences  among  these  approaches  to  measuring  state-of-the-art  in  their 
representation  of  technological  change  over  time.  In  the  development  of  analytic  models  and  estimating 
relationships,  there  are — Invariably — tradeoffs  between  (1)  the  desired  richness  of  detail  used  to  describe 
the  process  under  study,  and  (2)  the  need  to  economize  in  the  use  of  available  data  which  must  he  utilized 
in  calibrating  and  verifying  the  estimating  relationship.  The  several  alternative  approaches  to  measuring 
SOA  advance  represent  a variety  of  specific  tradeoffs  between  these  two  factors.  As  discussed  previously, 
several  of  the  approaches  permit  economical  use  of  available  historical  data  by  incorporating  the  entire 
data  sample  in  a "time  regression"  procedure.  However,  this  economy  is  realized  only  bv  giving  up  some 
"richness"  in  the  description  of  how  performance  variables  change  over  time.  In  the  time  regression  and 
delta  performance  procedures  there  is  an  implicit  assumption  that  the  relationships  among  the  technology 
variables  remain  consistent  over  time.  In  some  fields  of  application  this  is  not  the  case.  In  others. 


* 

The  regression  equation  of  time  as  a function  of  one  or  more  technology  variables  fixes  the  relationship 
among  these  various  elements.  However,  in  some  technical  fields  these  relationships  change  markedly  over 
time.  For  example,  some  years  ago  there  was  a consistent  effort  to  increase  the  power  output  of  fire  con- 
trol radars.  More  recently,  however,  technical  efforts  have  concentrated  on  other  features  in  response 
to  changes  in  operational  requirements.  We  have  noted  comparable  effects  in  studies  of  tactical  rocket 
motors . 

In  our  original  "surface"  approach,  the  SOA  surfaces,  which  are  defined  for  successive  time  intervals, 
need  not  move  outward  in  any  simple,  consistent  manner.  Each  surface  can  he  positioned  in  any  manner 
with  respect  to  previous  or  subsequent  surfaces,  as  may  he  indicated  hv  actual  technical  progress. 


2-17 


there  appears  to  be  a "smoothness"  in  the  implementation  of  technological  developments  for  which  these 
approaches  may  be  well  suited. 

During  these  studies  other  "operational"  considerations  have  come  to  light.  As  an  example,  the  first 
step  in  the  t ime- regress  ion  and  delta  performance  procedures  involves  an  equation  in  which  various  tech- 
nical and  performance  characteristics  are  related  to  calendar  time.  With  a properly  selected  set  of  char- 
acteristics, the  inference  is  that  the  resulting  estimating  function  depicts  the  time-changing  capability 
to  achieve  particular  performance  levels.  However,  with  another  set  of  characteristics,  the  equation  may 
reflect  the  time-changing  nature  of  mission  requirements,  rather  than  technical  capability.  In  this  in- 
stance, the  residuals  (Yg  - YAcj*jjAL)  w°uld  not  be  indicative  of  technologica  1 advance.  Tlv  *,uld  merely 
indicate  how  individual  systems  came  "before"  or  "after  their  time"  in  terms  of  changing  nh  .sion  requirements 
(or,  in  the  case  of  commercial  products,  changing  market  demand). 

To  give  a specific  example:  The  overall  size  of  computer  memory  is  governed  by  mission  goals,  and 
does  no'  represent  a particular  technological  constraint.  Use  of  this  mission-determined  parameter  in  tile 
time  regression  procedure  could  lead  to  specious  inferences  about  the  technological  advance  represented  in 
particular  machines. 

To  resolve  this  problem,  variables  should  be  chosen  to  reflect  technological  capability,  not  pure 
mission  parameters.  The  variables  should  be  given  in  terms  of  "efficiency  measures,"  such  as  calculations 
per  second,  watts  per  pound,  thrust  per  pound  . . . , rather  than  "absolute-scale  measures,"  such  as  total 
memory  size,  power  output,  thrust,  etc.  Efficiency  measures  represent  general  technological  goals  and  con- 
straints which  are  invariably  being  improved.  Hie  absolute-scale  or  size  indicators  are  more  directly  a 
function  of  mission  parameters,  e.g.,  large  missiles  or  computer  memories  are  not  indicative  of  technical 
capability  as  much  as  they  reflect  the  requirements  to  meet  operational  objectives.  Measures  of  changes 
over  time  in  these  parameters  more  often  reflect  changes  in  the  various  factors  affecting  operational  re- 
quirements (e.g.,  military  threat,  stated  goals  in  space  exploration,  business  requirements  in  commercial 
systems,  etc.).  In  these  instances  the  residuals  (Yg  - Y^-jO  cannot  be  interpreted  as  indicators  of  tech- 
nological advance. 

In  some  technical  areas  it  has  not  been  possible  to  characterize  the  state  of  technology  with  a re- 
duced set  of  measurable  parameters.  Fire  control  radars  and  radar-guided  missiles  are  prime  examples; 
conventional  measures,  such  as  accuracy  or  range,  do  not  represent  major  design  goals  in  current  systems. 
Rather,  emphasis  is  being  placed  upon  advanced  signal  processing  capabilities,  and  other  characteristics 
which  are  difficult  to  describe  as  scalar  parameters.  In  this  instance,  our  ability  to  account  for  tech- 
nological change  in  R&D  resource  analysis  is  limited. 

In  the  next  section,  technological  change  is  discussed  as  it  affects  procurement  costs. 

3.3  MANUFACTURING  TECHNOLOGY  (OR  PRODUCTIVITY)  INDICES 

3.3.1  Introduction 

It  is  standard  procedure  in  cost  analysis  to  adjust  recorded  costs  from  different  years  to  some 
constant-dol lar  basis,  e.g.,  FY  1975  dollars.  This  adjustment — the  price  level  index — normalizes  for 
variations  in  the  "purchasing  power"  of  the  dollar  in  obtaining  the  input  factors  (labor,  material,  etc.) 
of  production  or  development. 

For  many  equipments,  notably  electronics,  there  is  need  for  an  additional  index — an  "output  index" — 
to  normalize  for  year-to-year  changes  in  manufacturing  productivity,  i.e.,  in  the  ability  to  produce  a 
given  level  of  output  per  unit  cost.  This  point  is  amplified  in  the  following  paragraphs,  with  the  argu- 
ment being  made  that  failure  to  account  for  these  productivity  changes  is  a source  of  the  chronic  errors 
in  estimating  electronics  procurement  costs. 

3.3.2  A Productivity  or  Manufacturing  Technology  Index 

In  many  application  areas — most  notably  in  digital  computers  and  avionics — there  have  been  remarkable 
and  continuing  advances  in  technology  which  Impact  directly  upon  the  relationship  between  system  performance 
and  procurement  cost.  And  yet,  conventional  cost  analyses  do  not  take  these  advances  into  account,  an 
omission  which  can  result  in  serious  estimating  error. 

This  is  best  explained  by  illustration.  Consider  the  analyst  faced  with  the  problem  of  establishing 
a procurement,  cost-estimating  relationship  (CER)  for  computers.  His  task  is  to  identify  a functional  re- 
lationship between  computer  performance  (output)  and  the  cost  of  producing  that  capability.  He  might 
hypothesize  a CER  based  upon,  for  example,  processing  speed  (millions  of  instructions  per  second,  or  MIPS). 
Then,  historical  cost  and  performance  data  are  compiled  and  dollar  figures  are  carefully  adjusted  to 
FY  75  dollars.  However,  the  data  from  different  years  are  not  homogeneous  in  terms  of  the  production 
technology  underlying  the  performance-cost  relationship.  Early  computer  data  would  be  based  upon,  say, 

1969  production  technology.  More  recent  data  would  reflect  a later  technology,  and  the  aggregation  of 
these  figures  would  mix  time-varying  effects  of  advances  in  production  technology  with  the  "cross  sectional" 

* 

In  addition  to  computers,  this  same  problem  has  emerged  in  the  analysis  of  TWT  power  amplifiers.  This 
issue  has  also  arisen  in  studies  of  other  systems  such  as  tactical  missiles  and  fire  control  radars.  For 
example,  changes  in  mission  parameters  have  prompted  a shift  from  high-thrust  rocket  motors  to  longer 
sustained  burning.  Related  to  this  change  are  reduced  "torque"  demands  upon  missile  control  subsystems. 
Several  years  ago  there  was  considerable  emphasis  upon  higher  peak  powers  for  airborne  fire  control  radars. 
More  recently,  there  has  been  a shift  in  emphasis  to  signal  processing  techniques. 

In  these  examples,  the  time  regression  procedure — if  used  without  an  understanding  of  the  Interactions 
between  missions  requirements,  technology,  and  performance — could  be  totally  misleading. 


relationship  between  specific  performance  levels  and  cost  at  a given  point  in  time. 


Graphically,  these  relationships  can  be  viewed  in  Fig.  3.5.  If  the  analyst  has  three  data  points 
from  1969,  1971,  and  1975,  and  does  not  normalize  for  the  advance  in  technology,  he  may  face  the  following 
difficulty:  a plot  of  cost  (properly  normalized  for  price  level  changes)  versus  performance  may  look  like 
Points  1,  2,  and  3 (Fig.  3.5a).  Extrapolation  to  some  other  1975  equipment,  with  performance  X4  , would 
lead  to  a cost  estimate  Y4  . However,  the  data  are  really  "apples  and  oranges" — a mix  of  old  and  current 
technologies  (as  in  Fig.  3.5b).  To  properly  normalize  for  the  advance  in  manufacturing  technology.  Points 
1 and  2 must  be  adjusted  to  1*  and  2'  to  represent  the  cost  of  producing  the  1969  and  1971  computers  if 
they  were  actually  produced  in  1975.  With  proper  adjustment,  all  the  historical  data  would  be  placed  along 
the  same  technology  curve.  The  resulting  cost  estimate  would  be  Y^,  given  in  1975  dollars  using  1975 
technology. 

To  generalize  from  this  example:  There  are  numerous  application  areas  characterized  by  (1)  changes 
in  design  and  in  manufactur ing  technology  which  result  in  year-to-year  reductions  in  cost  per  unit  of  out- 
put (system  performance),  and  (2)  operational  requirements  for  continuing  increases  in  performance.  In 
these  circumstances,  the  failure  to  account  for  manufacturing  productivity  changes  represents  a source  of 
systematic  bias  (underestimation)  in  cost  analysis,  To  be  sure,  there  are  also  many  application  areas 
where  the  pace  of  technological  change — as  it  affects  cost — is  not  particularly  rapid,  and  historical  data 
covering  a span  of  years  can  be  used  without  this  productivity  adjustment.  But  in  specific  fields,  espe- 
cially avionics,  the  need  for  normalization  to  a particular  technology  seems  compelling.  A cost  estimate 
is  incomplete  without  specification  of  the  applicable  technology. 

Our  initial  inquiry  into  the  use  of  productivity  indices  was  in  an  analysis  of  avionics  computers 
for  the  USAF  Avionics  Laboratory ; ^8  subsequently  we  have  extended  the  investigation  in  a NASA-sponsored 
study  of  spacecraft  avionics.  In  the  computer  study  for  the  Air  Force  we  reviewed  the  intensive  literature 
indicating  the  rapid  changes  in  computer  productivity.  Sharpe^  depicted  a number  of  relationships,  as 
illustrated  in  Fig.  3.6.  Turn^O  indicated  the  historical  and  projected  data  shown  in  Fig.  3.7.  Seiler 
used  data  from  a Bell  Laboratories  study  in  formulating  a computer  technology-cost  index. ^1 

One  source  from  which  we  drew  extensively  was  a study  of  computer  indices  developed  for  the  Naval  Air 
Development  Center. ^2  The  NADC  study  is  a comparatively  detailed  inquiry  into  prospective  future  technolog- 
ical developments  in  circuitry,  components,  etc.  These  are  then  combined  into  indices  for  future  years 
from  a base  year  of  1970;  continued  rapid  improvements  are  forecast,  as  typified  in  Fig.  3.8.  The  index 
is  multiplied  by  1970  costs  to  establish  costs  using  a future-year  technology.  (Note  that  the  effect  is 
to  reduce  estimated  costs  for,  say,  1980;  while  the  effect  of  the  conventional  price-level  adjustment  of 
1980  dollars  would  be  an  increase  in  1980  cost.)  In  developing  a CER  for  avionics  computers  in  1974  tech- 
nology and  1974  dollars,  we  first  extended  the  indices  back  in  time  to  span  the  historical  data  base.  His- 
torical costs  were  then  adjusted  to  a 1974  technology  and  an  appropriate  CER  was  set  forth.  Not  surprising- 
ly, the  difference  in  the  estimating  equations  with  and  without  this  normalizing  index  was  significant. 

In  more  recent  studies  sponsored  by  NASA  and  by  the  U.S.  Air  Force,  we  have  been  examining  procedures 
for  deriving  empirical  measures  for  these  technology  indices.  Our  approach  is  to  incorporate  time  (year  of 
development  completion)  as  an  additional  independent  variable  in  a postulated  CER.  The  regression  coeffi- 
cient for  this  new  variable — derived  from  historical  data — may  be  interpreted  as  the  annual  change  in  cost 
due  to  shifts  in  the  estimating  relationship.^ 

Several  individual  subsystems  have  been  evaluated,  including  computers  and  communications  subsystems. 
For  example,  in  an  analysis  of  procurement  costs  for  avionics  digital  processing  units,  the  following 
equation  was  derived: 


InC  = 8.41  - .11  X.  + 0.249  X0  + 0.217  In  X^  + 0.274  In  X, 
12  3 4 


(3.4) 


Careful  analysts  have  recognized  the  pitfalls  in  trying  to  relate  summary  performance  measures  (such  as 
MIPS)  to  cost  without  accounting  for  differences  in  design.  But  this  accounting  has  generally  been  done 
in  only  a crude  manner,  e.g.,  stratifying  for  use  of  integrated  circuits  or  discrete  components.  Most 
changes  in  technology  seem  more  pervasive  and  consistently  changing  over  time.  Within  electronic  systems 
advances  In  solid-state  circuitry  have  actually  been  implemented  in  a gradual  ("smooth")  manner  over  time 
so  that  the  effects  of  these  advances  have  been  prolonged  and  continuous. 


In  estimating  the  cost  of  a reduced  performance  system,  there  would  be  a tendency  to  overestimate  the  cost. 
The  relationships  in  Fig.  3.5  are  exaggeraged  to  illustrate  the  impact  of  the  index. 

§ 

To  illustrate  in  terms  of  the  simplified  schematic  of  Fig.  3.5b — cost  (Y)  may  be  expressed  as  a function 
of  MIPS  (X)  as  follows: 

Y * a + bX 

However,  the  zero-intercept  is  a function  of  time:  a = a^  - a^t. 

Substituting,  we  get  Y = a^  - a^t  + bX  . In  this  illustration,  the  equation  becomes  the  new  model,  in- 
cluding both  the  general  CER  and  the  technology  index. 


where 


C » Cumulative  average  cost  at  Q * 100  in  thousands  of  1974  dollars 

Xj  * Year  of  development  completion  - 1900 

i 0,  if  microprogrammable 
2 ( l,  if  hard-wired,  synchronous 

3 

= 1 10  operations  per  second J [word  length,  bits] 

X.  * Number  of  distinct  instructions 
4 

The  first  variable  is  the  explicit  indicator  of  the  year-by-year  effect  of  advancing  technology.  Incorpor- 
ating this  indicator  in  the  estimating  equation  properly  accounts  for  this  significant  cost-determining 
factor  and  results  in  a more  accurate  portrayal  of  the  relationships  between  cost  and  the  other  performance 
measures.  The  empirically  derived  technology  indicator  can  be  used  to  forecast  future  costs  only  with  some 
assurance  that  historical  trends  will  continue.  The  analyst  may,  of  course,  revise  the  coefficient  in 
keeping  with  his  own  technological  forecast. 

Computers  have  exhibited  the  greatest  year-to-year  changes  in  manufacturing  technology — as  measured 
in  terms  of  performance  and  cost.  We  have  also  examined  other  equipments  such  as  transponders,  antennas, 
multiplexers,  and  radar  subsystems. 


The  preceding  discussions  have  touched  upon  several  of  the  methodological  refinements  we  have  been 
investigating  in  the  past  few  years.  This  work  is  obviously  incomplete  and  can  undoubtedly  be  refined  with 
additional  effort.  While  these  "mechanics”  of  accounting  for  technological  advance  are  "experimental"  we 
remain  firmly  convinced  that  the  underlying  phenomena  of  technological  change  and  its  impact  on  cost  are 
very  real  and  must  be  reflected  in  parametric  cost  analyses. 
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Figure  1.1.  Schematic  of  Cost  Analysis  Procedures 
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Figure  2.1.  Life  Cycle  of  a Weapon  System 
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THE  DEVELOPMENT  AND  IMPLEMENTATION 
OF  LIFE  CYCLK  COST  METHODOLOGY 
by 

T.D.  Kiang 

Manager,  Contract  Reliability  Engineering 
Bell-Northern  Research,  Ottawa,  Canada 


SUMMARY 

Bell-Northern  Research  has  developed  a life  cycle  cost  methodology  suitable  for  the  Canadian  Forces 
environments.  The  methodology  will  be  used  to  implement  the  life  cycle  management  system  established  by 
the  Canadian  Forces  Procedure  CFP  113.  This  paper  describes  the  concept  of  the  life  cycle  management 
system  and  the  life  cycle  costing  process.  The  unique  features  of  the  life  cycle  management  cost  model 
and  Its  capability  to  relate  cost  and  system  effectiveness  factors  are  discussed.  The  results  of  the 
model  application  to  the  AN/ARN-504  microtacan  are  presented. 


1.  INTRODUCTION 

The  concept  of  life  cycle  management  system  (LCMS)  was  initiated  in  1973  within  the  Canadian  Depart- 
ment of  National  Defence  (DND)  in  response  to  the  development  of  a comprehensive  defence  program  manage- 
ment system  (DPMS).  The  DPMS  is  aimed  at  providing  guidelines  for  managing  the  defence  services  program 
which  identifies  current  and  future  defence  resource  requirements.  The  LCMS  complements  the  DPMS  by 
providing  an  Integrated  approach  for  the  management  of  materiel  based  on  a life  cycle  concept.  The  LCMS 
calls  for  the  development  of  a methodology  suitable  for  DND  life  cycle  resource  evaluation  and  analysis. 

In  April  1976,  Bell-Northern  Research  (BNR)  was  awarded  a two-year  development  contract  by  DND.  The 
contract  was  divided  into  two  parts.  The  first,  completed  in  April  1977,  involved  the  development  of 
life  cycle  cost  (LCC)  methodology.  In  the  second  part,  a life  cycle  management  cost  model  (hereafter 
designated  as  DND  LCC  model)  was  constructed  by  establishing  the  relationship  between  cost  and  system 
effectiveness  factors.  The  model  was  validated  for  practical  application  using  the  microtacan  - an  air- 
borne electronic  navigation  system  carried  on  most  of  the  aircraft  in  the  Canadian  Armed  Forces.  The 
result  of  this  demonst rat  ion  has  presented  DND  with  a powerful  tool  for  comparative  engineering  evalua- 
tion  in  aid  of  management  decision-making.  The  methodology  will  help  life  cycle  materiel  managers  with- 
in the  Canadian  Forces  environments  in  the  procurement  of  new  systems,  for  optimization  studies  on  main- 
tenance, and  for  logistics  support  plans  for  existing  systems. 

Before  the  details  of  the  DND  LCC  model  are  presented,  the  intent  and  concept  of  the  LCMS  are 
briefly  discussed  to  provide  the  background  scenario  for  this  unique  Canadian  development  in  LCC  metho- 
dology. 


2.  LIFE  CYCLE  MANAGEMENT  SYSTEM 

2 . 1 Life  Cycle  Management  Concept 

Life  cycle  management  is  a concept  which  provides  for  the  management  of  all  activities  from  the  time 
a requirement  for  materiel  is  conceived  until  the  materiel  is  disposed  of.  It  is  defined  in  the  Canadian 
Forces  Procedure  CFP  113  (1)  as  "the  effective  management  of  all  activities  required  to  acquire  and  sup- 
port materiel  needed  for  operations,  from  the  time  of  its  initial  conception  to  the  time  of  its  disposal" 

The  principle  activities  performed  during  the  life  cycle  of  an  equipment  or  system  are  associated 
with  design,  evaluation,  acquisition,  installation,  maintenance,  support,  modification  and  disposal. 

Other  integral  activities  required  to  achieve  total  life  cycle  management  are  analysis  of  effectiveness 
and  life  costs. 

Ideally  it  is  desirable  to  have  one  manager,  who  can  be  held  accountable,  for  an  equipment  or  system 
throughout  Its  life  cycle.  The  length  of  the  life  cycle  for  most  Canadian  Forces  materiel  as  well  as  the 
movement  of  personnel  and  changes  of  organizat ional  structure  within  DND  precludes  the  achievement  of 
this  ideal  approach.  It  is,  therefore,  necessary  to  provide  a comprehensive  system  within  which  life 
cycle  management  can  be  achieved. 

The  LCMS  therefore  encompasses  and  consolidates  all  individual  life  cycle  management  activities.  It 
is  designed  to  ensure  that  any  activity  undertaken  is  within  stated  policy,  is  based  on  all  plans  and  de- 
cisions made  previously,  and  recognizes  future  requirements.  As  the  life  cycle  of  an  equipment  or  system 
spans  a considerable  length  of  time,  it  has  been  found  advantageous  to  divide  the  activities  into  stages. 

2 .2  The  Life  Cycle  Stages 

Although  the  life  cycle  of  an  equipment  or  system  can  be  divided  into  any  number  of  stages,  the  fol- 
lowing division  has  been  adopted  within  DND: 


(a)  Conception; 

(b)  Acquisition; 

(c)  In-Service;  and 

(d)  Disposal. 


The  conception  stage  encompasses  all  the  activities  which  are  necessary  to  develop  and  define  a 
means  of  meeting  a stated  requirement.  The  acquisition  stage  encompasses  those  activities  required  to 
acquire.  Install,  and  provide  future  support  for  the  equipment  or  system  selected  in  the  conception 
stage.  The  In-service  stage  is  normally  the  longest  and  encompasses  the  activities  directed  at  the  main- 
tenance, support,  and  modification  of  an  equipment  or  system  throughout  its  operational  life.  The  dis- 
posal stage  consolidates  the  activities  required  co  remove  the  equipment  or  system  and  its  supporting 
materiel  or  facilities  from  the  Canadian  Forces. 

2.1  The  Life  Cycle  Materiel  Manager 

The  person  responsible  for  coordinating  the  principle  life  cycle  management  activities  within  the 
LCMS  stages  Is  designated  as  a life  cycle  materiel  manager.  A life  cycle  materiel  manager  may  be  respon- 
sible for  one  or  more  svstems,  subsystems,  or  equipments  of  varying  complexity;  or  he  mav  be  designated 
for  the  management  of  common  item  materiel.  Figure  1 illustrates  the  relationship  between  life  cycle 
materiel  managers. 


Response  to  Ope  rations 


Figure  1 Relationship  between  Life  Cycle  Materiel  Managers 


2.4  Interface  with  Defence  Program  Management  System 

Figure  2 shows  the  interface  between  the  LCMS  and  the  DPMS. 

The  DPMS  commences  with  the  evaluation  of  a requirement  and  ends  with  the  placing  into  service  of  a 
resource  to  satisfv  that  requirement.  The  DPMS  provides  a system  for  identifying  a requirement  in  conso- 
nance with  policy,  developing  broad  optional  means  of  satisfying  the  requirement,  defiling  the  needed 
resources,  and  obtaining  approval  to  acquire  those  resources  and  placing  them  in  servu . . To  achieve 
this,  the  DPMS  is  structured  into  four  phases: 


<a! 

Policy 

Planning; 

(M 

Project 

Development ; 

(r) 

Projec t 

Definition;  and 

(<n 

Project 

Implement  at  ion , 

Policy  planning  is  the  process  of  determining  the  goals  of  PNO  and  describing  the  military  capabili- 
ties required  to  meet  them.  Project  development  is  determining  in  broad  terms  those  means  necessary  and 
feasible  to  enable  the  Canadian  Forces  to  achieve  agreed  levels  of  capability.  Project  definition  is  de- 
termining through  detailed  studv  and  analysis  the  resources  required  to  achieve  project  aims  within  im- 
posed constraints. 

The  conception  stage  of  the  LCMS  falls  into  the  first  three  phases  of  the  OPMS.  The  LCMS  acquisi- 
tion stage  coincides  with  the  DPMS  implementation  phase  which  is  concerned  with  the  implementation  of  the 
decisions  arrived  at  the  conclusion  of  project  definition.  The  degree  of  responsibility  and  the  extent 
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of  participation  In  the  two  systems  vary  from  project  to  project.  However,  once  a project  is  approved, 
the  LCMS  takes  over  the  prime  responsibility  to  manage  the  resources  through  its  in-service  life  and  its 
disposal . 

The  LCMS  provides  the  criteria  for  the  development  of  the  DND  LCC  model. 


Defence  Program  Management  System 


Life  Cycle  Management  System 
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Figure  2 Interface  between  Life  Cycle  Management  System  and 
Defence  Program  Management  System 


3.  LIFE  CYCLE  COST  METHODOLOGY 

3 . 1 Criteria  for  Model  Development 

The  LCC  methodology  was  developed  primarily  to  meet  the  needs  of  DND.  The  baseline  methodology  was 
to  provide  a generalized  model  that  relates  cost  and  system  effectiveness  factors  in  quantitative  terms. 
The  following  criteria  were  established  for  the  development  of  the  DND  LCC  model: 

(a)  The  model  must  form  a common  starting  point  for  LCC  evaluations. 

(b)  The  model  must  allow  flexibility  for  the  inclusion  of  additional  cost  contributing  factors 
peculiar  to  an  individual  project. 

(c)  The  model  must  allow  flexibility  for  the  deletion  of  cost  elements  that  are  not  applicable  in 
an  individual  project. 

(d)  The  model  must  be  constructed  so  that  complex  cost  functions  dependent  on  system  effectiveness 
parameters  (such  as  reliability,  maintainability  and  logistics  support)  can  be  computed  by  a 
programmable  facility. 

(e)  The  model  must  p.  ~o'ide  for  the  adjustment  of  future  expenses  to  a current  dollar  value  on  a 
specified  annual  discount  rate. 

Within  this  context,  rtNR  has  developed  a model  that  determines  optimum  system  values  with  respect  to 
LCC  and  an  important  system  effectiveness  parameter,  namely  operational  readiness.  To  permit  clarifica- 
tion of  the  technical  terms  used  in  the  model  development,  basic  definitions  were  established. 

3.2  Definitions 

(a)  Life  Cycle  Costing  is  the  process  of  economic  analysis  to  assess  the  total  cost  of  ownership, 
taking  into  consideration  the  constraints  associated  with  the  system  effectiveness  require- 
ments. 

(b)  Life  Cycle  Cost  is  defined  as  the  sum  of  one-time  initial  costs  incurred  in  the  life  cycle  of 
an  equipment  or  system  prior  to  operation  plus  the  ongoing  costs  incurred  during  its  operating 
life. 
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(c)  System  Effectiveness  is  the  probability  that  the  system  will  provide,  in  terms  of  the  resources 


3-4 


required,  and  as  specified,  either:  (i)  the  maximum  operational  performance  within  the  total 
cost  prescribed,  or  (ii)  the  required  value  at  the  lowest  total  cost. 


(d)  Operational  Readiness  is  the  probability  that,  at  any  point  in  time,  the  system  is  either  oper- 
ating satisfactorily  or  ready  to  be  placed  in  operation  on  demand,  when  used  under  stated  con- 
ditions including  allowable  warning  time.  Thus,  total  calendar  time  is  the  basis  for  computa- 
tion of  operational  readiness. 

3 . 3  Objective  Function  of  the  Model 

The  DND  LCC  model  is  designed  for  comparative  engineering  studies  in  projects  where  trade-offs  in 
life  cycle  resources  of  systems  are  possible  and  permissible.  The  model  represents  a set  of  mathematical 
expressions  relating  a general  and  flexible  profile  simulating  real  life  system  operation.  Due  to  the 
non-linear  and  complex  relationship  that  exists  between  system  operational  readiness  and  its  LCC,  a mar- 
ginal allocation  approach  using  Lagrange  multipliers  has  been  adopted  for  optimization  purposes.  This 
approach  employs  an  interactive  technique  for  optimal  allocation  of  resources. 

The  objective  function  of  the  model  is  to: 

(a)  maximize  system  operational  readiness  for  a given  life  cycle  cost  constraint,  or 

(b)  minimize  life  cycle  cost  to  meet  a given  system  operational  readiness  requirement. 

Mathematical  expressions  for  the  above  objective  function  can  be  formulated  as: 


Max  As , subject  to  LCC  <^C  ------------------------------  - (2) 

Min  LCC,  subject  to  As  > A -------------------------------  (2) 


where  As  * System  operational  readiness 
LCC  = Life  cycle  cost 
C = Life  cycle  cost  constraint 
A = System  operational  readiness  target 

Detailed  mathematical  expressions  and  their  derivations  are  documented  in  a BNR  report,  Mathematical 
Model  [21. 

3.4  Model  Structure 


The  structure  of  the  DND  LCC  model  is  presented  in  Figure  3.  The  modular  structure  of  the  model 
provides  flexibility  in  dealing  with  particular  requirements  of  individual  systems,  and  in  performing 
different  types  of  analyses. 

Each  block  shown  in  Figure  3 represents  an  element  of  LCC  or  operational  readiness.  For  conveni- 
ence, the  blocks  are  arranged  in  a set  of  numerical  series.  Blocks  in  the  100-series  refer  to  elements 
of  LCC;  blocks  in  the  200-series  refer  to  elements  of  operational  readiness.  Data  requirements  for  each 
block  are  detailed  in  the  LCC  Methodology  report  [3], 

The  double  arrows  superimposed  on  the  DND  LCC  model  in  Figure  3 indicate  some  of  the  more  important 
relationships  between  cost  elements  and  effectiveness  elements;  these  arrows  are  not  meant  to  be  exhaus- 
tive for  there  are  many  more  direct  and  indirect  relationships. 

The  model  is  capable  of  dealing  with  any  system  that  consists  of  one  or  more  prime  equipments  that 
are  situated  at  one  or  more  locations;  in  addition,  the  model  takes  into  account  the  maintenance  and 
logistics  support  (MLS)  system  available  to  support  the  prime  equipment. 

To  facilitate  the  computational  process,  mathematical  expressions  developed  for  the  DND  LCC  model 
are  programmed  in  Fortran  IV  using  an  IBM  VM  370/168  computer  system.  The  interactive  feature  of  the 
computerized  model  permits  rapid  access  to  sensitivity  analyses  and  parametric  evaluations  of  LCC  or 
operational  readiness. 

3.5  Computation  of  Operational  Readiness 

To  compute  operational  readiness,  a parameter  of  system  effectiveness,  the  DND  LCC  model  takes  into 
account  the  operational  readiness  of  all  prime  equipment  associated  with  the  system.  In  a system  com- 
posed of  several  prime  equipments,  the  prime  equipment  may  be  distributed  over  several  locations.  The 
problem  is  first  considered  location  by  location.  For  each  location,  the  model  computes  the  operational 
readiness  of  prime  equipment  at  that  location  based  on: 

(a)  factors  peculiar  to  that  location  which  contribute  to  operational  readiness,  and 

(b)  factors  arising  from  the  maintenance  and  logistics  support  system  common  to  all  locations. 

Once  the  operational  readiness  of  prime  equipment  at  each  location  has  been  determined,  the  model 
can  then  compute  overall  system  operational  readiness,  which  is  the  prime  system  effectiveness  parameter 
of  concern  in  the  DND  LCC  model. 

The  operational  readiness  of  prime  equipment  at  each  location  is  computed  as  a function  of  mean 
uptime  and  mean  downtime.  The  calculation  of  mean  uptime  takes  into  account  mean-t ime-between-f a 1 1 urea 
(MTBF),  the  utilization  rate  (t.e.  ratio  of  operating  hours  to  operable  hours),  the  number  of  prime 
equipments  in  the  system,  and  the  mean-t ime-between-scheduled-ma 1 ntenanre  (MTBSM).  The  calculation  of 
mean  downtime  takes  into  account  mean  corrective  maintenance  time  and  mean  scheduled  maintenance  time. 
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The  greatest  Impact  on  the  result  Is  from  the  mean  corrective  maintenance  time,  which  depends  on  the  time 
needed  for  fault  diagnosis  and  checkout,  the  time  to  replace  the  faulty  Item,  administrative  time,  and 
supply  delay  time  (i.e.  the  time  required  to  obtain  a spare  item).  Supply  delay  time  depends  upon  the 
MLS  system. 

The  model  will  compute  supply  delay  time  for  each  location  of  prime  equipment.  In  addition.  It  can 
compute  the  optimum  number  and  allocation  of  spares  to  be  held  at  each  location  In  the  MLS  system  if 
these  values  have  not  been  predetermined. 
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Figure  3 DND  LCC  Model 


3.6  Maintenance  and  Logistics  Support  System 

The  DND  LCC  model  makes  detailed  analysis  of  the  maintenance  and  logistics  support  (MLS)  system 
available  to  support  the  prime  equipment.  This  MLS  system  encompasses  all  activity  by  the  repair  facili- 
ties and  spares  pools  at  various  locations. 


3-6 

The  many  variables  in  the  MLS  system  determine  the  time  the  prime  equipment  must  wait  for  a spare 
(t.e.  supply  delay  time,  Block  260  of  Figure  3).  Thus,  the  MLS  system  affects  downtime,  and  ultimately 
system  operational  readiness.  In  addition,  possible  changes  in  the  MLS  system  to  increase  or  reduce 
supply  delay  time  will  affect  LCC. 

The  major  portion  of  LCC  often  tends  to  be  due  to  recurring  costs  during  the  in-service  stage. 

Thus,  LCC  evaluations  related  to  the  MLS  system  can  strongly  influence  LCC  and  achievable  operational 
readiness.  The  problem  is  much  more  than  a question  of  how  many  spares  are  needed  for  the  initial  pro- 
curement. The  model  can  help  to  determine  the  effect  on  LCC  and  system  operational  readiness  of  deci- 
sions relating  to:  repair  policies,  number  and  type  of  test  equipment  to  be  used  by  repair  facilities, 
manpower,  etc. 

Figure  4 illustrates  a typical  configuration  of  a comprehensive  MLS  system.  At  each  prime  equipment 
location,  the  failed  equipment  is  restored  back  to  service  by  the  test  and  repair  facility  by  replacing 
faulty  line  replaceable  units  (LRU)  with  a spare  LRU.  This  spare  LRU  is  provided  by  the  MLS  system.  The 
faulty  LRU  is  sent  to  the  MLS  system  for  repair. 


PE:  prime  equipment 


Figure  4 Typical  Configuration  of  Comprehensive  Maintenance  and 
Logistics  Support  (MLS)  System 


The  MLS  system  described  in  the  model  consists  of  at  most  three  levels  of  facilities  depending  on 
the  maintenance  and  support  plan  for  a particular  prime  equipment.  Each  MLS  facility  consists  of  a test 
and  repair  facility  and  a spares  pool.  The  test  and  repair  facility  and  the  spares  pool  need  not  neces- 
sarily be  located  at  the  same  place.  The  test  and  repair  facilities  are  dedicated  to  effecting  repairs 
that  cannot  be  performed  at  the  preceding  level  of  MLS  facilities. 

The  general  repair  policy  is  determined  by  the  hierarchy  of  the  prime  equipment  and  the  cause  of 
failure.  This  general  repair/sparing  policy  incorporated  in  the  DND  LCC  model  for  each  level  of  the  MLS 
system  is  summarized  in  Table  l.  A typical  operation  of  the  MLS  system  is  illustrated  in  Figure  S . 

3.7  Computation  of  Life  Cycle  Cost 

The  model  computes  LCC  as  a function  of  non-recurring  and  recurring  costs.  Non-recurring  or  one- 
time costs  include,  where  applicable:  R&D,  hardware  acquisition,  operations,  maintenance,  logistics  sup- 
port, administrat ion , and  disposal.  Recurring  costs,  computed  on  an  annual  basis,  include:  finance 
charges,  operations,  maintenance,  logistics  support,  and  administration. 
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These  costs  are  adjusted  by  a discount  rate  to  convert  the  value  of  future  dollars  to  present  dol- 
lars and,  if  desired,  to  account  for  the  effects  of  inflation.  These  calculations  are  extended  over  the 
expected  lifetime  of  the  system,  or  the  period  under  study. 


3-7 


Table  1 Summary  of  Repair/Sparing  Policy 
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Type  I:  Restoration  of  Failed  Item  Is  Effected  By  Replacement  of  Lower  Level  Components 
Type  II:  Restoration  of  Failed  Item  Is  Effected  By  Repair  of  Item 
PE:  Prime  Equipment 
LRU:  Line  Replaceable  Unit 


3.8  Relationship  Between  System  Effectiveness  and  LCC 

The  double  arrows  superimposed  on  the  DND  LCC  model  in  Figure  3 indicate  some  of  the  more  important 
relationships  between  cost  elements  and  system  effectiveness  elements;  these  arrows  are  not  meant  to  be 
exhaustive  for  there  are  many  more  direct  and  indirect  relationships.  The  matrix  of  Table  2 provides  a 
more  comprehensive  indication  of  some  of  the  relationships  that  can  exist  between  elements  of  LCC  and 
elements  of  system  effectiveness. 

In  general  terras,  system  effectiveness  can  be  improved  or  reduced  by: 

(a)  decisions  in  R&D  that  influence  system  inherent  reliability  and  maintainability,  and 

(b)  decisions  relating  to  the  maintenance  and  logistics  support  provided  to  the  system. 

In  both  cases,  decisions  in  (a)  and  (b)  influence  cost  as  well  as  system  effectiveness.  For  example 
a decision  to  use  high-reliability  parts  or  to  incorporate  built-in-test  equipment  in  the  system  should 
result  in  improved  reliability  and  operational  readiness;  at  the  same  time,  these  decisions  will  probably 
result  in  higher  acquisition  costs  but  may  well  produce  lower  overall  costs  over  the  life  of  the  system. 

Sensitivity  studies  can  help  identify  those  relationships  in  which  small  changes  in  input  values 
have  a significant  effect  on  system  effectiveness  and  LCC;  thus  efforts  can  be  concentrated  in  these 
areas  which  have  the  greatest  potential  for  cost  savings  and  improved  effectiveness.  Sensitivity  studies 
are  performed  by  exercising  the  DND  LCC  model  over  a range  of  input  values/decisions;  in  this  way  it  may 
be  found  that  some  types  of  input  data  have  a negligible  effect  on  the  model  output  over  the  range;  other 
types  of  input  can  be  identified  where  small  changes  In  input  produce  substantial  changes  on  the  model 
output • 

3.9  Assumpt ions 

The  LCC  methodology  developed  for  DND  is  generalized  in  nature.  It  is  designed  to  accommodate  most 
systems  and  equipments  encountered  in  the  Canadian  Forces  environments  for  LCC  application.  To  meet  such 
a broad  scope  of  application,  some  assumptions  are  made  and  incorporated  in  the  DND  LCC  model.  The  fol- 
lowing summarizes  these  assumptions  and  limitations  that  are  inherent  in  the  model  construction. 

(a)  All  failures  are  random 


Failures  are  assumed  to  follow  a Poisson  process.  Where  failures  do  not  strictly  follow  a 
Poisson  process,  the  model  provides  results  that  in  most  cases  are  within  reasonable  bounds  for  LCC 
and  operational  readiness.  However,  piecewise  linear  approximation  has  been  used  successfully  to 
simulate  wearout  failure  characteristics  for  all  practical  purposes. 
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Figure  S Typical  Operation  of  MLS  System 

( b ) F.very  prime  equipment  failure  is  caused  by  failure  of  a line  replaceable  tin  l_t 

The  DND  LCC  model  operates  on  the  understanding  that  a spare  LRU  will  he  required  to  restore 
operation  of  the  prime  equipment.  This  assumption  neglects  the  possibility  that  some  fraction  of 
prime  equipment  failures  may  he  repairable  without  the  installation  of  a spare  l.RU,  e.g.  repaired  hv 
adjustment,  etc.  This  assumption  applies  only  to  prime  equipment  failures.  In  the  case  of  LRU 
failures  and  module  failures,  the  model  considers  two  types  of  restoration:  hv  repair  (Type  II), 
and  by  replacement  with  a spare  component  (Type  I).  The  impact  of  this  assumption,  depending  upon 
the  fraction  of  failures  of  this  type,  is  that  the  DND  LCC  model  will  over-estimate  the  number  of 
spare  LRUs  required,  and  will  allow  waiting  time  for  obtaining  the  unneeded  spare;  that  is,  the  DND 
LCC  model  output  will  to  some  extent,  be  more  pessimistic  than  real  life  situation. 


Table  2 Cost  and  SK  Relationship  Matrix 
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(c ) No  lateral  transfer  of  spares  occurs  between  spares  pools 

All  spares  pools  are  assumed  to  obtain  their  spares  from  the  preceding  level  of  spares  pool  In 
the  logistics  support  hierarchy.  In  real  life,  a spares  pool  may  on  occasion,  for  expediency,  re- 
quest a spare  from  another  spares  pool  of  the  same  level  as  itself.  The  effect  of  this  real  life 
practice,  if  permitted,  would  he  to  decrease  the  waiting  time  for  a spare  needed  to  make  a repair. 
Thus  the  DND  LCC  model  output  does  not  allow  for  this  occasional  and  irregular  shortcut  and  hence 
could  he  pessimistic  to  some  degree;  the  real  life  experience  should  he  the  same  or  N't  ter. 

( d ) System  effectiveness  is  measured  as  operational  readiness 

The  DND  LCC  model  is  constructed  to  measure  svstem  effectiveness  In  terms  of  operational  read! 
ness.  This  model  does  not  consider  operational  performance  or  mission  reliability;  if  these  para- 
meters are  important  on  a particular  project,  the  life  cycle  materiel  manager  must  take  them  into 
account  in  his  decision-making  process. 


(e)  The  DND  LCC  model  requires  input  data 
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The  DND  LCC  model  deals  In  quantitative  data.  To  exercise  this  model,  the  life  cycle  materiel 
manager  must  be  able  to  provide  input  data.  This  input  data  can  be  estimated  statistically  based  on 
historic  experience  or  comparisons  with  similar  equipment;  input  data  can  be  based  upon  actual  in- 
formation, or  it  could  be  based  on  the  personal  judgment  of  the  life  cycle  materiel  manager,  or  even 
pure  guesses.  The  operation  of  the  DND  LCC  model  does  not  depend  upon  the  quality  of  input  data  but 
merely  upon  the  availability  of  data  on  which  to  base  its  computat ions . In  this  respect,  the  accur- 
acy of  the  output  produced  by  the  model  will  depend  upon  t he  quality  of  data  provided.  The  better 
the  estimates  or  data,  the  more  accurate  will  be  the  DND  LCC  model  output.  However,  by  conducting 
sensitivity  analyses,  the  life  cycle  materiel  manager  can  identify  those  data  items  that  have  a 
strong  impact  on  the  model  output,  and  those  data  items  that  have  a lesser  or  negligible  impact  on 
output.  In  this  way,  he  can  determine  which  data  items  warrant  the  greatest  effort  in  obtaining 
accurate  estimates,  and  which  data  items  can  be  estimated  with  less  accuracy  or  even  ignored. 

( f ) The  DND  LCC  model  considers  quantitative  data 

The  model  does  not  deal  with  intangibles  but  in  values  that  can  be  expressed  quant itatlvely . 

Any  applicable  intangibles  must  be  weighed  by  the  life  cycle  materiel  manager  in  the  decision-making 
process . 

(g)  The  DND  LCC  model  is  intended  for  comparative  studies 

This  model  should  be  used  only  for  comparative  studies.  One  indirect  advantage  of  using  a 
model  for  comparative  studies  is  that  it  can  be  useful  even  when  accurate  input  data  are  not  avail- 
able; if  inputs  are  estimated  in  a consistent  manner,  even  though  they  may  err  in  absolute  terms, 
the  model  can  compute  relative  advantages  or  disadvantages  of  various  options. 


4.  APPLICATION 

4.1  Application  of  DND  LCC  Model  to  Life  Cycle  Management  System 

The  DND  LCC  model  is  Intended  as  a tool  for  carrying  out  comparative  engineering  studies  of  options 
and  tradeable  requirements  available  at  different  times  or  stages  in  the  life  cycle  of  a system  or  equip- 
ment. To  do  this,  the  DND  LCC  model  evaluates  each  alternative  in  terms  of  LCC  and  operational  readi- 
ness. Figure  6 illustrates  typical  applications  for  the  DND  LCC  model  throughout  the  various  stages  in 
the  life  cycle  of  a system  or  equipment.  As  the  life  cycle  progresses,  the  alternatives  available  de- 
crease. The  model  helps  to  compute  quantitative  data  useful  in  assessing  trade-offs  between  cost  and 
effectiveness  factors.  It  presents  a powerful  decision  tool  for  the  life  cycle  materiel  managers. 
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Figure  6 


Application  of  LCC  Model  to  Life  Cycle  Management  System 
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4.2  A Decision  Tool  for  Life  Cycle  Materiel  Managers 

The  DND  LCC  model  Is  highly  adaptable.  Its  ultimate  worth  to  the  life  cycle  materiel  manager  will 
be  determined  by  his  vision  in  applying  it,  i.e.  by  the  types  of  analyses  and  trade-off  studies  he  de- 
cides to  perform  and  his  timing  in  conducting  these  studies.  The  model  cannot  make  decisions  but  it  can 
help  answer  questions.  It  is  up  to  the  life  cycle  materiel  manager,  however,  to  ask  the  right  questions: 
what  will  be  the  effect  on  cost  (or  effectiveness) 

...  if  expected  system  life  is  doubled?  or  halved? 

...  if  the  pipeline  distance  between  2nd  and  3rd  line  is  increased? 

...  if  the  system  MTBF  is  improved? 

...  if  more  sophisticated  test  equipment  are  provided? 

The  tree  diagram  in  Figure  7 illustrates  how  the  DND  LCC  model  can  be  used  by  the  life  cycle  materi- 
el manager  in  the  decision-making  process.  The  tree  starts  with  a system  effectiveness  requirement;  the 
first  level  of  branching  represents  the  choice  between  possible  prime  equipments.  For  each  possible 
prime  equipment,  a second  level  of  branching  depicts  the  test  equipment  that  might  be  used  to  support  the 
prime  equipment.  The  third  level  of  branching  represents  possible  repair  policies,  i.e.  the  particular 
MLS  system  chosen,  including  the  many  variables  in  this  system,  e.g.  number  of  lines  in  the  system,  num- 
ber of  facilities  at  each  line,  distance  between  the  lines,  etc. 


System  Effectiveness 


\ 


Allocation  of 
Spares 


Level  of  Branching 


Optimum 
Life  Cycle  Cost 


Figure  7 LCC  Decision  Tree 


The  final  destination  of  each  path,  depending  upon  the  choice  made  at  each  branching  ixiint,  is  the 
optimum  number  and  allocation  of  spares  needed.  This  destination  can  be  determined  bv  the  DND  LCC  model, 
based  upon  a requirement  for  minimizing  LCC  or  a requirement  for  maximizing  system  effectiveness. 

At  the  conception  stage,  the  possible  decision  paths  include  all  branches.  Once  prime  equipment  has 
been  selected,  however,  some  branches  are  eliminated,  but  manv  choices  still  remain.  Sometimes,  the  test 
equipment  that  may  be  selected  depends  on  the  choice  of  prime  equipment,  i.e.  the  prime  equipment  manu- 
facturer mav  recommend  a test  equipment  that  is  mandatory  or  at  least  desirable  for  use  with  the  prime 
equipment • 
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The  branches  relating  to  the  repair  policy  and  MLS  system  to  a large  extent  remain  available 
throughout  the  life  of  the  equipment.  The  DND  LCC  model  can  be  used  In  the  early  stages  of  system  life 
(conception,  acquisition,  beginning  of  in-service  life)  to  evaluate  possible  MLS  systems.  This  evalua- 
tion can  usefully  be  repeated  at  intervals  throughout  the  In-service  life:  to  verify  the  MLS  system 
selected  earlier  based  on  updated  input  data;  to  determine  the  Impact  of  proposed  equipment  modifica- 
tions; to  determine  the  impact  of  a proposed  change  In  effectiveness  requirements,  etc. 

As  the  equipment  nears  the  end  of  Its  useful  life,  the  DND  LCC  model  can  evaluate  options  concerning 
possible  modifications  or  overhaul  to  the  prime  equipment  to  extend  its  useful  life  versus  the  option  of 
retiring  the  equipment  and  replacing  it  with  new  equipment. 

A . 3 Typical  Applications 

The  following  list  names  some  of  the  applications  of  the  DND  LCC  model  that  are  typical  for  each 
stage  of  the  LCMS: 

(a)  During  conception 

i)  sensitivity  studies  to  determine  what  parameters  have  a major  impact  on  cost  or  system 
effectiveness 

li)  analysis  of  options  relating  to  system  configuration,  system  reliability  and  maintaina- 
bility, or  operational  readiness  of  the  system  with  respect  to  LCC 

ill)  impact  of  possible  combinations  of  manpower  availability  and  skill  level  requirements  on 
LCC 

iv)  evaluation  of  effect  of  maintenance  policies  on  LCC  and  operational  readiness 

v)  comparison  of  R&O  contractor  versus  organic  maintenance  within  DND  in  terras  of  LCC 

vi)  Influence  of  test  equipment  on  LCC  and  operational  readiness 

vii)  evaluation  of  systems  that  are  operationally  equivalent  (but  different  m other  respects) 
in  terms  of  LCC. 

( b)  During  acquisition 


i) 

evaluation  for  procurement  of  a system  given  a cost  constraint 
requirement 

or 

operat ional 

readiness 

11) 

optimization  of  MLS  system 

111) 

evaluation  of  maintenance  policy  in  terras  of  LCC 

lv) 

optimization  of  spares  requirement  given  an  operational  readiness 
constraint 

requi rement 

or  cost 

v) 

optimization  of  support  resources,  such  as  location  of  spares, 
transportation 

level  of  maintenance, 

vl) 

forecasting  of  support  expenditures  over  the  life  cycle. 

(c)  In-service 


1)  verification  of  analyses  carried  out  during  the  conception  or  acquisition  stage 

11)  evaluation  of  proposed  repair  and  overhaul  contracts 

ill)  evaluation  of  Impact  on  LCC  and  operational  readiness  of  proposed  software  or  hardware 
modifications 

lv)  evaluation  of  effect  on  LCC  of  changes  in  MLS  system 

v)  Impact  on  LCC  and  operational  readiness  of  proposed  modifications  or  overhaul  to  extend 
system  life. 

(d)  At  disposal  stage 

i)  impact  on  LCC  of  extending  system  life  versus  replacing  system  with  new  acquisition 

li)  evaluation  of  Impact  on  LCC  of  available  options  in  disposal,  transportation,  documenta- 
tion, and  administration  to  phase  out  system. 

It  should  be  noted  that  the  exercise  of  the  DND  LCC  model  does  not  produce  decisions;  it  only  helps 
the  life  cycle  materiel  manager  make  more  knowledgeable  decisions. 

A .A  The  Decision-Making  Process 

In  applying  the  LCC  methodology,  the  decision  process  involves  the  following  procedural  steps: 
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(a)  Definition  of  the  problem 

(b)  Selection  of  alternatives  that  meet  performance  requirements 

(c)  Definition  of  input  data  requirements 

(d)  Acquisition  of  data 

(e)  Execution  of  model 

(f)  Interpretation  of  output 

(g)  Sensitivity  analysis  for  cost  drivers 

(h)  Evaluation  of  results  with  other  intangible  and  uncontrollable  parameters 

(i)  Final  decision. 

Figure  8 shows  a simplified  flow  diagram  for  the  decision-making  process. 


Figure  8 Decision-Making  Flow  Diagram 
5.  A CASE  STUDY  ON  THE  AN/ARN  504  MICROTACAN 


The  purpose  of  the  case  study  is  to  demonstrate  the  practical  application  of  the  DND  LCC  model,  and 
to  validate  the  LCC  methodology  developed  for  DND  operation.  The  case  study  consists  of  a baseline  study 
and  three  comparative  studies  to  fully  illustrate  the  features  and  capability  of  the  DND  LCC  model  as 
applied  to  the  various  stages  of  the  system  life  cycle. 


The  system  selected  by  DND  as  a vehicle  to  demonstrate  the  application  of  the  DND  LCC  model  is  the 
Micro-Elec t conic  Airborne  TACAN,  AN/ARN  504.  It  must  be  emphasized  that  the  use  of  this  system's  data 
and  configuration,  Including  the  support  system,  is  for  demonstration  purposes  only.  Results  presented 
herein  make  no  implications  concerning  past  decisions,  or  future  actions,  due  to  the  fact  that  the  base- 
line data  provided  by  DND  has  been  expanded  to  fully  demonstrate  the  model's  capability. 

5.2  Scope  of  the  Case  Study 

(a)  Baseline  study 

This  study  deals  with  several  hundred  microtacan  equipments  deployed  at  a number  of  Canadian 
Forces  bases.  The  prime  equipments  are  supported  by  a three-echelon  maintenance  and  logistics 
support  system.  There  are  16  first  lines  supported  by  4 second  lines  and  1 third  line  depot.  Data 
on  prime  equipments  and  the  support  system  were  input  to  the  model  and  used  as  a baseline  in  the 
three  comparative  studies. 
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( b)  Study  1 ~ comparison  of  prime  equipments  for  source  selection 

This  study,  conducted  during  the  conception  or  acquisition  stage,  compares  three  different 
microtacan  equipments  for  source  selection.  The  trade-offs  are  between  cost  and  reliability. 

( c ) Study  2 ~ comparison  of  alternate  maintenance  and  logistics  support  plans 

This  study,  usually  conducted  during  early  ln-servlce  stage,  deals  with  the  comparison  of  al- 
ternate maintenance  and  logistics  support  for  the  microtacan  equipment. 


( d)  Study  3 ~ comparison  of  options  for  replacement  of  aging  equipment 

This  study  is  usually  conducted  as  the  equipment  approaches  its  disposal  stage.  It  examines 
the  options  relating  to  replacement/modernizat ion  of  aging  equipment.  This  study  is  done  to 
evaluate  the  economic  viability  of  extending  equipment  service  life. 


Each  of  these  studies  was  performed  using  the  computerized  l)ND  LCC  model.  Details  of  the  system 
operational  scenario,  data  requirements,  options  considered,  analysis  results  and  conclusions  are  pre- 
sented in  the  following  sections.  Complete  case  study  on  the  AN/ARN  504  microtacan  is  documented  in  a 
BNR  report  on  LCC  Case  Studies  14). 

5.3  Baseline  Study  V ‘ « 

i • 

This  section  describes  aspects  of  the  case  study  which  are  common  and  form  a baseline  for  assessing 
alternat Ives. 


(a)  The  system  under  study 


Figure  9 illustrates  the  three  LRUs  of  the  microtacan  system.  Cost,  weight  and  failure  rate 
data  are  given  in  Table  3.  The  Receiver-Transmitter  (R/T)  LRU  consists  of  its  chassis  plus  ten  re- 
placeable modules  (excluding  the  bandpass  filter  and  isolator  which  are  passive  devices,  non-repair- 
able,  and  rarely  require  replacement).  The  Adapter  LRU  consists  of  its  chassis  plus  two  replaceable 
modules.  The  Control  LRU  is  not  further  divisible  into  modules,  and  must  be  returned  to  the  repair 
and  overhaul  (R&0)  contractor  for  repair.  The  Mount,  while  referred  to  as  an  LRU,  has  not  been  in- 
cluded in  this  study  due  to  its  passive  nature. 


Figure  9 Micro-Electronic  Airborne  TACAN , AN/ARN  504 
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Table  3 Cost,  Weight,  and  Failure  Rate  Data  of  the  AN/ARN  504  Microtaean 


LINE  REPLACEABLE  UNIT 
(LRU) 

MODULE 

COST 

(S) 

WEIGHT 

(kg) 

FAILURE  RATE 
(x  10”6) 

RECEIVER  TRANSMITTER 

21 ,000 

10.7 

5000 

P/A  Modulator 

2 ,7  SO 

405 

Frequency  Generator 

3,350 

458 

Decoder 

2,350 

417 

Internal  Control 

1 ,850 

769 

Bearing  Computer 

2,4  50 

909 

Range  Computer 

2,100 

540 

Circulating  Monitor 

750 

48 

Self  Test 

950 

196 

R/T  Chassis 

2,000 

50 

Power  Supply 

1 ,550 

804 

Receiver 

2,870 

148 

ADAPTER 

5,000 

6.6 

800 

Range  Coupler 

2,700 

200 

Bearing  Coupler 

3,000 

200 

Adapter  Chassis 

1,000 

400 

CONTROL 

1 ,200 

0.9 
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( b)  Operation  and  support  configuration 

Figure  10  illustrates  the  operation  and  support  configuration  for  the  sixteen  Canadian  Forces 
bases  where  the  microtaean  equipments  are  located. 

These  sixteen  1st  Line  facilities  are  supported  by  four  2nd  Line  facilities  which  in  turn  are 
supported  by  a single  3rd  Line  maintenance  depot.  With  the  exception  of  the  P/A  modulator  module  in 
the  R/T  LRU,  all  other  modules  are  serviced  by  the  R&0  contractor  at  the  3rd  line  depot. 

The  statistics  concerning  number  of  prime  equipments  (aircraft  in  this  case),  operating  hours 
per  month,  and  distances  from  supporting  facilities,  remain  constant  throughout  the  case  study. 

( c)  Spares  flow  and  repair  activity 

There  are  basic  differences  in  the  spares  flow  and  repair  activity  between  each  of  the  micro- 
tacan  LRUs.  The  baseline  statistics  used  throughout  these  studies  concerning  order  and  shipping 
time,  repair  time,  and  percentage  of  occurrences,  are  given  on  each  of  the  diagrams  illustrating  the 
respective  activity  for  each  of  the  LRUs.  Figure  11  depicts  baseline  activity  for  the  Control  LRU, 
while  Figures  12  and  13  cover  the  Adapter  and  R/T  LRUs  respectively. 

In  Figure  11,  it  can  be  seen  that  the  Control  LRU  is  sent  to  2nd  Line  for  failure  verification 
and  confirmation,  and  then  either  returned  to  stores  if  good,  or  sent  to  R&O  for  repair.  Replace- 
ment Controls  are  ordered  as  Indicated  to  replenish  the  respective  spares  stock. 

Figure  12  concerns  the  Adapter  LRU,  whose  activity  parallels  that  of  the  Control  LRU  until  con- 
firmation of  a fault  at  2nd  Line.  Here,  as  there  are  modules  involved,  repair  can  be  effected  by 
module  replacement. 

With  the  R/T  LRU,  faults  are  liable  to  occur  a small  percentage  of  the  time  (5 Z in  this  case) 
which  cannot  be  repaired  by  module  replacement.  This  type  of  fault  is  defined  as  "Type  II",  whereas 
"Type  I"  faults  refer  to  faults  which  module  replacement  can  rectify.  This  is  illustrated  in  Figure 
13  where  the  spares  flow  and  repair  activity  for  the  R/T  is  depicted. 

( d)  Input  data 

Table  4 lists  the  input  data  used  in  this  case  study. 

The  data  used  were  the  best  available  from  DND  at  the  time  the  study  was  performed.  It  was 
felt  that  using  the  best  available  data  would  make  the  case  study  more  realistic,  although  it  should 
be  emphasized  that  the  purpose  of  the  case  study  was  to  exercise  the  DND  LCC  model  rather  than  to 
draw  conclusions  concerning  the  microtaean  system. 

(e)  Output 

Figure  14  shows  a plot  of  LCC  vs  OR  for  the  baseline  data  inputs.  This  plot  was  obtained  by 
varying  the  requirement  for  OR  values  between  0.7  and  0,998,  and  obtaining  the  resulting  optimum 
values  of  LCC. 


Table  4 Case  Study  Baseline  Data 


MODEL 

BLOCK 

REFERENCE 

INPUT  DESCRIPTION 

INPUT  DATA 

1 

UNITS 

101 

Period  of  Study  In  LCC  Analysis 

20 

Years 

102 

Discount  Rate 

10 

* /An  num 

10b 

Non-Recurring  HurJware: 

Acquisition  Costs 

27,300 

Dollars 

112 

Non-Recurring  Operating  Costs: 
Documentation  and  Software 

30 

Dollars 

122 

Non-Recurring  Maintenance  Costs: 
Documental  ion  and  Software 

900 

Dollars 

123 

Transportation  Cost  Rate  of  Test 

Equipment 

1 

$ /Ton- km 

124 

Non-Recurring  Maintenance  Costs: 

Repair  Facilities 

40.000 

Do  l lars 

132 

Transportat ton  Cost  Rate  for  Spares 

i 

S /Ton- km 

171 

Annual  Maintenance  Cost: 

Consumables  for  Repair 

1 ,400 

' S/Year 

172 

Loaded  Labour  Rate  for  Repair  at 

PE  Test /Repair  Facility 

14 

S/Hour 

172 

Loaded  Labour  Rate  for  Repair  at 

1st  Line  MLS  Facility 

14 

| $/Hour 

172 

Loaded  Labour  Rate  for  Repair  at 

2nd  Line  MLS  facility 

14 

5 /Hour 

172 

Loaded  Labour  Rate  tor  Repair  at 

3rd  Line  MLS  Facility 

H 

$/Hour 

212 

Average  Oj. rational  Time  of  PE/Month 

40 

Hours /Month 

212 

Average  Time  PE  Required  to  be 
Operable/Montb 

720 

Hours /Month 

231 

Mean  Test  Time  at  PE  Test /Repair 

Facility 

2 

Hours 

233 

No  Fault  Found  Ratio 

1/3 

F ract Ion 

240 

Average  LRl'  Replacement  Time  it 

PE  Test/Repair  Facility 

i 

Hours 

266 

Number  of  LRUs  in  PE 

3 

Number 

1 2” 

Average  Actual  Repair  Time  at  2nd  Line 

6 

Hours 

1 “ 

Average  Actual  Repair  Time  at  3rd  Line 

20 

Hours 

ED — 

I--”  l — 

ED — 
ED- 

ED 1 

ED — 


-ED 


-ED— 


0- 


Figure  10  Operation  and  Support  Configuration 
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Figure  11  Baseline  Repair  Activity  for  Control  LRU 
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Figure  12  Baseline  Repair  Activity  for  Adapter  LRU 
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Figure  16  Life  Cycle  Cost  Summary  (Baseline  Case) 
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5.4  Study  1 ~ Comparison  of  Prime  Equipments  for  Source  Selection 
(a)  Description 


Study  1 deals  with  the  selection  of  three  alternative  microtacan  systems  on  a LCC  basis.  The 
repair  and  maintenance  policy  is  equivalent  for  all  three  alternatives. 


This  study  could  take  place  in  either  the  conception  or  acquisition  stage  of  the  LCMS.  The 
basic  approach  is  the  same  in  either  stage,  only  the  depth  of  analyses  would  differ  depending  upon 
the  data  availability  and  accuracy. 

The  alternative  microtacan  systems  will  be  referred  to  as  Alternates  A,  B and  C,  with  Alter- 
nate A being  the  baseline  system  as  described  in  the  previous  section. 

The  following  data  changes  apply  to  Alternate  B and  Alternate  C. 

Alternate  B.  Costs  for  LRUs  and  modules  are  120%  of  the  baseline  figures  shown  in  Table  3. 

Failure  rates  are  50%  of  the  baseline  figures  shown  in  Table  3. 

Alternate  C.  Costs  for  LRUs  and  modules  are  150%  of  the  baseline  figures  shown  in  Table  3. 

Failure  rates  are  50%  of  the  baseline  figures  shown  in  Table  3. 

K 

The  baseline  support  system  is  applied  in  all  three  alternatives. 

( b)  Results 

The  results  obtained  from  exercising  the  DND  LCC  model  with  the  input  data  described  for  Alter- 
nates A,  B and  C are  shown  graphically  in  Figure  17.  The  data  computed  by  the  model  for  an  OR  of 

0.80  were  analyzed  and  the  results  summarized  in  Table  5.  L 
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Table  5 Life  Cycle  Cost  Breakdown  (Study  1) 


ALTERNATIVE 

NON-RECURRING 

(NR) 

RECURRING 

(LCC-NR) 

TOTAL  LCC 

A (baseline) 

B 

C 

$ 14.52M 
$ 15.16m 
$ 19.10m 

S 4.93M 

S 2.37M 
$ 2.43M 

S 19.45M 

S 17.53M 

S 21.53M 

$M 


operational  readiness 

Figure  17  Plot  of  LCC  vs  OR  (Study  1) 
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( c ) Conclusions 

It  can  be  concluded  from  this  particular  study  that  a limit  exists  to  the  costs  which  should  be 
incurred  for  hardware  acquisition,  in  order  to  achieve  lower  hardware  failure  rates,  for  any  speci- 
fied value  of  equipment  operational  readiness. 


From  the  LCC  breakdown  in  Table  5 and  related  analyses,  the  following  conclusions  can  also  be 
drawn: 

l)  The  recurring  costs  are  virtually  halved  by  halving  the  failure  rates. 

il)  While  the  initial  spares  are  less  for  reduced  failure  rates,  the  cost  is  offset  to  a varying 

extent  by  an  increase  in  hardware  acquisition  cost. 

It  should  be  noted  that  only  by  plotting  a family  of  curves  can  a proper  appreciation  be  obtained  of 
these  effects. 

5 . 5 Study  2 - Comparison  of  Alternate  Maintenance  and  Logistics  Support  Plans 
(a)  Descript  ion 

Study  2 demonstrates  the  use  of  the  DND  LCC  model  in  making  comparisons  between  alternative 
maintenance  and  logistics  support  systems.  The  microtacan  equipment  details  are  as  described  in  the 
baseline  case. 

The  alternative  support  systems  are  referred  to  as  Alternate  A and  Alternate  B. 

Alternate  A.  The  support  system  is  as  described  for  the  baseline  case. 

Alternate  B.  The  support  system  is  depicted  in  Figure  18.  Essentially  all  2nd  Line  mainten- 

ance has  been  eliminated,  and  the  proposition  is  made  to  ship  all  failed  items  (LRUs  in  this  case) 
directly  to  3rd  Line  (i.e.  the  R&O  contractor).  Figure  19  illustrates  the  repair  activity  which 
will  take  place  under  this  alternative  support  system. 


lit  Line  3rd  Line 


Figure  18  Operation  and  Support  Configuration  (Study  2) 
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Figure  19  Repair  Activity  for  Alternative  B (Study  2) 


( b)  Results 

The  results  obtained  in  this  study  from  applying  alternative  support  systems  are  graphically 
displayed  in  Figure  20. 


SM 


operational  readiness 

Figure  20  Plot  of  LCC  vs  OR  (Study  2) 

The  data  computed  by  the  model  for  an  OR  of  0.80  were  analyzed  and  the  results  summarized  in 
Table  f>. 

Table  f>  Life  Cycle  Cost  Breakdown  (Study  2) 


NON-RECURRINC 

RECURRING 

ALTERNATIVE 

(NR) 

(LCC-NR) 

TOTAL  LCC 

A (baseline) 

S 14 . 52M 

S 4.93M 

S 1 9 , 4 SM 

B 

$ 16.N8M 

S 3.1.™ 

S 20.33M 

( c)  Conclusions 

From  the  LCC  breakdown  in  Table  b and  related  analyses,  It  can  be  concluded  that  while  the  re- 
curring costs  of  manpower  for  maintenance  are  reduced,  by  eliminating  the  2nd  Line  maintenance 
activity,  the  extra  (non-recurring)  costs  incurred  for  initial  spares  outweigh  this  factor  and  make 
the  proposed  support  system  more  expensive  on  a LCC  basis  over  the  upper  range  of  OR  (0,7  and  up). 
At  lower  values  of  OR  the  reduced  amount  of  initial  spares  required  reduces  the  advantage  of  one 
maintenance  policy  over  the  other.  It  is  not  anticipated  that  values  of  OR  less  than  70?,  would  be 
of  practical  interest. 

Study  3 - Comparison  of  Options  for  Replacement  of  Aging  Equipment 

(a)  Descript  ion 

Study  3 examines  two  alternatives  in  a hypothetical  situation  where  the  equipment  and  support 
system  are  as  described  in  the  baseline  case.  The  point  in  time  at  which  this  study  is  assumed  to 
take  place  is  after  IS  years  of  in-service  operation  out  of  an  originally  planned  life  of  20  years 
(see  Figure  21).  At  this  point  consideration  is  being  given  to  continuing  the  equipment  use  for 
another  ten  years  (i.e.  five  years  more  than  originally  planned). 
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Figure  21  Extension  of  Equipment  Life  (Study  3) 

It  is  assumed  that  the  power  supply  module  is  now  only  achieving  half  of  its  originally  speci- 
fied MTBF.  All  other  modules  meet  their  OTBF  requirements.  Because  of  this  reduced  MTRF  of  the 
power  supply,  the  microtacan  operational  readiness  is  lower  than  desired. 

The  two  alternate  courses  of  action  being  considered  are  referred  to  as  Alternate  A and  Alter- 
nate B,  and  are  described  below: 


Alternate  A.  Purchase  additional  spare  power  supply  modules  to  raise  the  microtacan  operation- 
al readiness. 

Alternate  B.  Replace  all  existing  power  supply  modules  with  a new  design,  having  twice  the 
MTBF  specified  for  the  existing  model,  but  with  a cost  increase  of  20%.  An  additional  $100K  onetime 
cost  is  also  added  to  account  for  the  design  and  retrofit  of  the  replacement  module. 

(b)  Results 

Results  of  study  3 are  plotted  in  Figure  22. 

(c ) Conclusions 

It  may  be  concluded  from  this  study  that  Alternate  A,  although  more  costly  in  the  mid-range  of 
OR,  i not  significantly  greater,  and  has  the  advantage  of  verified  data  to  support  it.  Alternate  B 
may  be  more  costly  in  the  long  run  if  the  failure  rate  again  does  not  hold  to  the  specified  value. 

b.  CONCLUSIONS 

The  case  study  on  the  AN/ARN  504  microtacan  has  successfully  demonstrated  the  application  of  the  DNP 
LCC  model  to  the  Canadian  Forces  operating  environments.  The  success  of  this  demonstration  has  led  to 
the  award  of  several  further  contracts  to  BNR , ranging  from  LCC  evaluation  of  aircraft  engine  availa- 
bility to  comparison  of  replacement  policies  for  a vehicle  fleet. 

The  LCC  methodology  was  developed  primarily  to  meet  the  needs  of  DND.  Bv  standardizing  on  this 
methodology,  DND  obtains  a common  starting  point  for  LCC  evaluations.  The  methodology  could  be  extended 
to  non-DND  applications  to  conduct  LCC  and  system  effectiveness  analyses  on  complex  systems  such  as  power 
distribution,  transportation,  and  communication  networks.  The  methodology  can  also  be  adapted  to  speci- 
fic requirements  for  sensitivity  analyses,  evaluation  of  reliabilitv  improvement  warranties,  engineering 
economy  studies,  and  other  appl icat ions . 


operational  readiness 

Figure  22  Plot  of  LCC  vs  OR  (Study  3) 
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SUMMARY 


As  operation  and  support  costs  of  the  complex,  sophisticated  weapon  systems  of  today 
have  begun  to  far  exceed  the  original  acquisition  costs,  there  has  been  increasing  pres- 
sure to  develop  and  implement  Life  Cycle  Cost  (LCC)  methodology  in  the  analysis  of  such 
systems.  To  use  such  a methodology,  one  must  have  available  valid  models  which  in- 
corporate acquisition  costs  and  operation  and  support  costs  over  some  defined  useful 
life  period.  Many  models  have  been  developed,  but  it  has  only  been  recently  that  they 
have  begun  to  be  applied.  The  purpose  of  this  paper  is  to:  (1)  acquaint  you  with  a few 
of  the  representative,  available  models;  (2)  discuss  the  methods  by  which  the  models 
were  developed;  (3)  analyze  the  models  in  terms  of  their  shortcomings  and  sensitivities, 
and  (4)  discuss  recent  applications  of  these  models  in  avionics  procurements.  A brief 
bibliography,  containing  information  on  additional  models,  is  included.* 

INTRODUCTION : The  Life  Cycle  costs  of  a system  include  its  acquisition  costs  and  its 

operating  and  maintenance  (O&M)  costs  for  its  useful  life.  From  this  may  be  subtracted 
any  salvage  value  recovered  at  the  end  of  its  useful  life.  The  latter  is  usually  neg- 
ligible and  will  not  be  addressed  in  this  paper.  We  will  address,  in  order,  acquisition 
cost  models,  O&M  cost  models,  a model  examining  reliability  as  a capital  investment, 
considerations  of  inaccuracy  in  LCC  estimates  and  some  recent  applications  of  LCC  models 
in  Air  Force  procurements. 

ACQUISITION  COST  MODELS:  Acquisition  cost  models  are  usually  characterized  by  two  types: 
development  cost  models,  and  production  cost  models.  Acquisition  cost  models  are 
usually  empirical  in  nature  (although  there  are  some  exceptions) . They  are  generally 
developed  directly  from  available  data;  the  inherent  correlation  and  relationships  among 
the  data  variables  are  allowed  to  frame  the  form  of  the  resulting  model.  One  obvious 
modeling  approach  is  the  use  of  multiple  regression  in  which  one  develops  a mathematical 
relationship  between  acquisition  costs  and  the  significant  variables  affecting  acqui- 
sition costs.  Regression  analysis  is  a widely  used  modeling  technique  and  hence,  merits 
some  discussion. 

SIMPLE  LINEAR  REGRESSION:  We  shall  illustrate  the  use  of  regression  techniques  by  using 
simple  linear  regression  to  derive  a model  relating  the  cost  of  airborne  radio  sets  to 
their  weight.  This  is  based  on  an  intuitively  appealing  thesis  that  as  the  weight  of  a 
system  increases  it  becomes  more  costly.  It  is  unlikely  that  weight  alone  will  prove  a 
satisfactory  indication  of  cost,  but  this  will  be  checked  in  the  course  of  the  analysis. 
Our  input  data  is  presented  in  Table  1. 


TABLE  1 

TEN  AIRBORNE  RADIO  COMMUNICATION  SETS 
COST  ($)  WEIGHT  (LB) 


22,200 

90 

17,300 

161 

11,800 

40 

9,600 

108 

8,800 

82 

7,600 

135 

6,800 

59 

3,200 

68 

1,700 

25 

1,600 

24 

* The  authors  above  are  particularly  indebted  to  the  authors  of  the  following  reports 
and  publications,  extracts  from  which  form  a major  portion  of  this  lecture  text. 

(1)  "An  Introduction  to  Equipment  Cost  Estimating",  C.  A.  Batchelder,  et  al,  The 
Rand  Corporation,  RM-6103-SA,  December  1969. 

(2)  "Reliability  Acquisition  Cost  Study",  S.D.  Mercurio  and  C.  W.  Shaggs,  RADC-TR- 
77-334,  June  1973. 

(3)  "Reliability  Acquisition  Cost  Study  (II),  Hughes  Aircraft  Company,  RADC-TR-75- 
270,  November  1975. 

(4)  "Reliability  Trade-offs  for  Unit  Production  Cost",  T.W.  Butler,  June  1978, 
in  printing  as  an  RADC  Technical  Report. 

(5)  "Reliability  as  a Capital  Investment",  A.  Coppola,  Proceedings  1974  Annual 
Reliability  and  Maintainability  Symposium,  Los  Angeles,  California,  29-31  January  1974. 
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The  form  of  the  relationships  between  cost  and  the  explanatory  variable(s)  depends 
on  the  problem.  It  may  reflect  either  an  underlying  physical  law  or  a structural  re- 
lationship. When  no  particular  functional  form  is  suspected,  a simple  (two-variable) 
linear  model  is  frequently  used  to  describe  the  relationship  between  two  variables.  In 
this  case,  the  equation  of  the  model  is 

y = a + bx,  (1) 

where  y is  the  dependent  variable  and  x is  the  explanatory  variable.  The  symbols  a and 
b are  the  constant  and  coefficient,  respectively,  of  the  equation  estimated  from  the 
data.  Here  y could  represent  the  cost  of  a radio  communication  set  and  x could  represent 
the  weight.  If  it  is  assumed  that  b is  greater  than  zero,  the  model  indicates  that 
heavier  equipment  will  cost  more  than  lighter  equipment.  When  the  values  of  a and  b are 
known,  it  is  possible  to  compute  y (cost)  for  any  given  value  of  x (weight). 


LEAST- SQUARES  ESTIMATING:  Given  Eq . (1),  the  basic  problem  in  the  first  phase  of  the 
regression  analysis  is  to  derive  estimates  of  the  parameters  a and  b.  The  standard  pro- 
cedure is  the  method  of  least- squares . The  values  of  a and  b are  determined  by  the 
requirement  that  the  sum  of  the  squares  of  the  deviations  of  the  sample  observations 
from  the  estimated  line  will  be  at  a minimum.  Symbolically,  this  minimum  is  expressed 
as 

min  S (y.  - y.)2,  (2) 

i=l  1 1 

Where  y^  is  the  ith  observation  and  y^  is  the  value  of  y^  estimated  from  the  equation 

yt  = a + bx^  (3) 

A A A A 

The  carets  over  a and  b indicate  that  a and  b are  least-squares  estimates  of  the 

true  but  unknown  values  of  a and  b.  Thus  y^  is  the  least-squares  estimate  of  y.  and 

the  term  (y^  - y . ) indicates  the  difference  between  each  observed  y;  and  each  1 

corresponding  estimated  yi.  This  is  illustrated  in  Fig.  1,  which  snows  the  actual  (y) 
and  estimated  (y)  value  of  the  dependent  variable  that  corresponds 
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FIG.  1 - DEVIATION  OF  ACTUAL  VALUE  FROM  ESTIMATED  VALUE  AND  SAMPLE  MEAN 


to  a specific  value  of  the  explanatory  variable  x.  The  line  shown  in  Fig.  1 is  the  line 
that  represents  Eq.  (3).  All  of  the  estimated  values  of  vj  fall  on  this  line.  The 
vertical  distance  from  point  A to  point  B is  the  difference  between  the  actual  value  (y) 
and  the  estimated  value  (y) . The  summation  of  all  such  differences  that  are  squared 
(as  illustrated  in  Eq.  (2))  is  the  quantity  to  be  minimized  in  estimating  the  line. 
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The  minimum  value  for  this  sum  is  satisfied  by  substituting  gq . (3)  in  Eq  . (2) , 
taking  the  partial  derivatives  of  Eq.  (2)  with  respect  to  a and  b,  and  setting  the  re- 
sults equal  to  zero.  This  process  yields  two  equations  that  are  called  normal  equations 
and  that  can  be  solved  for  a and  £>: 

Ey=na+bEx, 


Exy=aEx+b£x2, 


where  y = cost  of  airborne  radio  equipment  in  thousands  of  dollars, 
x = weight  of  airborne  radio  equipment  in  pounds, 
n = number  of  items  in  the  sample, 

Z = summation  (e.g.,  Z y = the  sum  of  all  y's). 

to  solve  the 


WEIGHT 

xy 


1.998.0 

2,785.3 

472.0 

1,036.8 

721.6 

1.026.0 
401.2 

217.6 
42.5 
38.4 


8,739.4 

s are  expressed  in  thousands 
of  dollars.  When  the  values  from  Table  2 are  substituted  in  the  normal  equations,  the 
following  expressions  are  obtained  for  the  sample  data  points  (n  = 10): 

90.6  = 10a  + 792b, 

8739.4  = 792a  + 81, 540b. 

Solved  simultaneously,  these  equations  give 

a = 2.477, 

b = .083, 

and  thus  from  Eq . (3) 

y = 2.477  + . 083x  (4) 

The  line  represented  by  this  equation  is  shown  in  Fig.  2 as  the  solid  line  with  the 
actual  observations  plotted  as  dots.  The  extent  of  the  dispersion  of  the  observations 
relates  inversely  to  the  usefulness  of  the  line  as  a tool  for  estimating  the  values  of  y 
from  the  values  of  x.  The  greater  the  dispersion  of  observed  values  of  y about  the  line, 
the  less  accurate  the  estimates  that  are  based  on  the  line  are  likely  to  be  The  measure 
of  the  dispersion  about  the  regression  line  is  called  the  standard  error  of  estimate  (SE) 
of  the  equation  and  is  shown  by  the  dashed  lines. 

One  measure  of  dispersion  in  a collection  of  data  points  is  called  the  variance.  The 
variance  is  defined  as  the  sum  of  the  squared  distances  to  each  of  the  data  points  from 
a central  reference  point  divided  by  the  degrees  of  freedom  (df),  which  equal  the  number 
of  independent  bits  of  information  contained  in  the  sample.  (In  analyzing  the  data 
that  are  given  in  Table  1,  the  degrees  of  freedom  equal  (n  - 2),  i.e.,  the  number  of 
observations  n less  the  number  of  constraints,  1 each  for  a and  b. 

In  least- squares  procedures,  the  central  point  of  reference  for  calculating  the 
variance  of  each  variable  is  its  sample  mean,  which  causes  the  least-squares  line  to 
have  the  property  of  passing  through  the  means  of  the  variables  used  to  estimate  the 
line.  This  characteristic  is  shown  in  Fig.  1;  it  can  be  verified  by  dividing  both  sides 
of  the  first  normal  equation  by  n.  since  the  sample  mean  of  any  variable  v is 


Table  2 contains 

the  numerical 

values 

and  totals  requi: 

TABLE  2 

DATA  FOR  REGRESSION 

ANALYSIS  OF  COST 

X 

1 

90 

22.2 

8,100 

161 

17.3 

25,921 

40 

11.8 

1,600 

108 

9.6 

11,664 

82 

8.8 

6,724 

135 

7.6 

18.225 

59 

6.8 

3,481 

68 

3.2 

4,624 

25 

1.7 

625 

24 

1.6 

576 

792 

90.6 

81,540 
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FIG.  2 - REGRESSION  LINE  AND  STANDARD  ERROR  OF  ESTIMATE 


By  referring  to  Fig.  1,  it  can  be  seen  that  the  total  distance  from  to  y for  any 
observation  on  y is  the  distance  from  C to  B.  The  sum  of  all  such  distances  squared 
and  divided  by  the  degrees  of  freedom  is  called  the  total  variance  of  y: 


Total  variance  of  y = X 


(Yi  - y)2 
n - 1 


(6) 


The  distance  from  C to  A indicates  the  amount  of  the  total  deviation  of  y from  y which 
is  exglained  by  the  estimating  relationship.  Consequently,  the  sum  of  the  distances 
from  y to  the  line,  squared  and  divided  by  the  degrees  of  freedom,  is  called  the 
explained  variance: 


Explained  variance  of  y 


(7) 


The  remaining  distance  from  A to  B is  the  residual  or  unexplained  deviation  from  y^  to 
y,  or  the  unexplained  variance: 


Unexplained  variance  of  y 


(8) 


The  standard  error  of  estimate  is  defined  as  the  square  root  of  the  unexplained 
variance  of  the  y's: 


SE  = 


For  the  equation  y = 2.477  + 083x,  the  standard  error  of  estimate  is  $5,808.  This 
value  has  been  plotted  above  and  below  the  regression  line  in  Fig.  2.  The  interpreta- 
tion and  significance  of  these  results  will  be  discussed  in  connection  with  the  use  of 
prediction  intervals. 

In  comparing  one  SE  with  another,  it  is  useful  to  compute  a relative  standard  error 
of  estimate.  One  such  measure  is  the  coefficient  of  variation  (CV) , which  relates  the 
SE  to  the  mean  of  the  sample  y's: 


'J 


l - yt>‘ 

n - 2 


(9) 


U 


r: 
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y (10) 

Continuing  the  analysis  of  the  data  in  Table  1,  the  mean  of  the  y's  is  $9,060.  There- 
fore, the  value  of  CV  is 


$5,808  _ 64i 

$?tW0  ' ‘ 


This  value  is  high.  Although  the  question  of  reliability  of  an  estimating  equation  is 
relative  to  the  context  in  which  the  equation  is  to  be  used,  a value  at  least  as  small 
as  10  to  20  percent  for  the  coefficient  of  variation  is  desirable. 

The  standard  error  of  estimate  gives  a measure  of  the  magnitude  of  the  unexplained 
variance.  Another  related  measure  of  dispersion  is  given  by  the  coefficient  of  deter- 
mination that  shows  the  proportion  of  total  variance  accounted  for  by  the  estimating 
relationship: 

r2  = Coefficient  of  determination  = Explained  variance 

Total  variance 


= 1 - Unexplained  variance. 
Total  variance 


(ID 


When  all  the  observed  points  in  the  sample  are  on  the  least-squares  line,  the  co- 
efficient of  determination  equals  1 and  there  is  no  unexplained  or  residual  variance. 

As  the  proportion  of  total  variance  that  remains  unexplained  increases,  the  coefficient 
of  determination  approaches  zero.  The  square  root  of  the  coefficient  of  determination 
is  called  the  correlation  coefficient.  Correlation  has  no  substantive  meaning  unless 
both  the  dependent  and  explanatory  variables  are  assumed  to  be  normal  random  variables. 
The  ordinary  assumption  in  using  regression  analysis  for  developing  estimating  relation- 
ships is  that  only  the  dependent  variable  is  random.  Consequently,  it  is  not  considered 
good  practice  for  the  correlation  coefficient  to  be  used  in  documenting  the  results  in 
this  particular  application  of  regression  analysis.  The  inclusion  of  the  correlation 
coefficient,  however,  causes  no  serious  problem  since  it  is  simply  the  square  root  of 
the  coefficient  of  determination.  When  analysts  review  the  results,  they  can  easily 
calculate  the  latter  from  the  former.  Since  the  coefficient  of  determination  is  always 
in  the  range  between  zero  and  one,  its  square  root  will  always  be  larger,  except  at  the 
boundary  points  of  zero  and  one. 

The  coefficient  of  determination  for  Eq.  (4)  is  .325,  which  is  relatively  low  and 
further  substantiates  the  evidence  that  weight  alone  is  not  a good  predictor  of  the  cost 
of  airborne  radio  communication  equipment. 

PREDICTION  INTERVALS:  The  procedure  for  calculating  the  prediction  interval  for  a simple 
regression  is  as  follows . For  a given  value  of  the  explanatory  variable,  say  x,  the 
estimating  equation  is  used  to  obtain  a predicted  value  of  the  dependent  variable: 

y = a + bx.  (12) 

The  prediction  interval  puts  a boundary  around  y: 

y * A e/2.  (13) 


There  is  a certain  level  of  confidence  (1  - e)  that  the  cost  of  a set  weighing  x will 
be  in  that  interval. 

Values  for  e/2  rather  than  e are  used  since  y is  to  be  bounded  on  both  sides.  The 
values  of  e can  be  divided  by  two  since  under  the  assumptions,  the  probability  dis- 
tribution about  y is  normal  and  therefore  is  symmetrical.  In  statistical  terminology, 
a two-tailed  t distribution  for  constructing  the  intervals  is  used. 


In  the  case  of  simple  regression,  a 100(1  - e)  - percent  prediction  interval  for  an 
estimated  value  of  the  dependent  variable  can  be  constructed  as  follows: 


where 


(14) 


t/2 


(SE)t 


t/2 


4 


n + 1 
n 


£ (x ^ - x)2 


(15) 


and  where  SE  = 

t - the 
e/2 

n = the 
x = the 


the  standard  error  or  the  estimating  equation  from  which  y was  obtained, 
value  obtained  from  a table  of  t-values  for  the  e/2  significance  level, 
size  of  the  sample, 

specified  value  of  the  explanatory  variable  used  as  a basis  for 


r 


obtaining  y, 

x = the  mean  of  the  x's  in  the  sample, 

_ o 

(x.  - x)  = the  sum  of  the  squared  deviations  of  the  sample  x's  from  their  sample 
1 mean. 


When  the  estimating  equation  derived  previously  is  used,  the  cost  of  a communica- 
tions set  weighing  100  lbs  is  estimated  at  $10,777.  To  establish  around  this  value  a 
90-percent  prediction  interval  (i.e.,  one  with  a 10-percent  level  of  significance),  the 
necessary  data  are 


Z (x 


SE 


e/2 

t 

n 

df 

x 

x 

x)2 


5.808, 

0.1, 

0.05, 

1.86, 

10, 

8, 

100  lbs, 

79.2  lb, 
18,813.6  lb. 


By  substituting  these  data  in  Eq.  (15),  solving  for  A /2,  and  multiplying  by  1000, 
we  obtain 

A£/2  = $11,447. 

Therefore,  for  x = 100  lb,  the  90-percent  prediction  intervals  in  dollars  are 

y + Ae/2  = $10,777  + $11,447. 

The  percentage  100(  1 - e)  is  the  confidence  level  of  the  prediction  intervals,  which 
means  that  if  repeated  observations  on  the  cost  of  communications  sets  that  weigh  100 
lb  were  taken,  100(1  - e)  percent  of  the  time  these  observations  would  lie  within  the 
range  set  by  the  100(1  - t)  prediction  intervals.  This  is  the  only  sense  in  which  a 
level  of  confidence  can  be  associated  with  prediction  intervals.  It  is  erroneous  to 
infer  that  there  is  a 100(1  - e)-percent  probability  that  the  actual  value  for  any 
particular  case  will  lie  within  the  interval. 

Further,  prediction  intervals  are  valid  outside  the  range  encompassed  by  the  sample 
data  that  are  used  to  generate  the  estimating  relationship  and  the  interval  only  if 
the  estimating  relationship  is  itself  valid  outside  that  range.  For  example,  if  there 
were  occasion  for  the  line  to  curve  up  or  down  or  if  a discontinuity  in  the  form  of  a 
discrete  jump  in  cost  occurred  for  weights  outside  the  sample  range,  this  fact  would  not 
be  reflected  in  the  prediction  interval.  Thus,  it  must  be  clearly  indicated  when  the 
intervals  are  used  for  estimates  based  on  values  outside  the  sample  range. 

This  prediction  interval  procedure  can  be  repeated  for  other  values  of  x and  the 
results  plotted  to  obtain  a 90-percent  prediction  interval  band  around  the  regression 
line,  as  shown  in  Fig.  3.  In  this  case,  the  90-percent  confidence  region  is  fairly 
wide  because  of  the  relatively  large  standard  error  of  this  equation.  The  formula  for 
the  prediction  interval  is  such  that  the  width  of  the  interval  is  sensitive  to  the  size 
of  the  standard  error;  large  standard  errors  indicate  that  much  of  the  cost  variation 
in  the  observed  data  is  unexplained  by  the  equation. 
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FIG.  3 - THE  90-PERCENT  PREDICTION  INTERVAL  BAND 
FOR  ESTIMATED  COSTS  BASED  ON  SAMPLE  DATA 
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TABLE  10 

GOODNESS  OF  FIT  RESULTS  FOR  RELIABILITY  COST  (Cont'd) 
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In  using  regression  analysis,  it  must  be  noted  that  the  statistical  significance  of 
regression  relationships  does  not  necessarily  imply  existence  of  a causal  relationship. 
The  following  excerpt  from  an  Institute  of  Defense  Analyses  (IDA)  memorandum  illustrates 
the  importance  of  this  distinction  in  cost  analysis:* 

Frequently  during  cost  effectiveness  studies,  the  distinction  between  a 
"causation"  cost  model  and  a "correlation"  cost  model  is  overlooked.  A simple 
example  will  be  used  to  illustrate  the  distinction  between  the  two  types  of 
cost  models  and  show  how  a sensitivity  analysis  performed  with  a correlation 
cost  model,  rather  than  a causation  model,  can  lead  to  erroneous  conclusions. 

Example:  Estimate  the  cost  of  assembling  a piece  of  hardware.  The  assembly 

consists  merely  of  bolting  various  elements  together.  The  overwhelming  majority 
of  the  cost  of  the  assembly  process  is  the  salary  paid  to  the  men  who  do  the 
bolting.  Careful  analysis  of  all  the  available  cost  data  might  yield  a cor- 
relation cost  model  given  by  Equation  1. 

C = a x w (1) 

where  w is  the  total  weight  of  all  the  bolts  that  go  into  the  assembly, 

C is  the  cost  of  the  assembly, 
a is  a regression  coefficient. 

By  all  of  the  various  statistical  measures  of  goodness  of  fit.  Model  1 is  a 
valid  prediction  equation. 

The  causation  cost  model  is  given  by  Equation  2. 

C = k x h x n (2) 

where  k is  the  hourly  wages  of  the  assemblers, 

h is  the  number  of  hours  it  takes  to  fasten  and  bolt, 

n is  the  number  of  bolts  used  in  the  final  assembly, 

C is  the  cost  of  the  assembly. 

It  should  be  noted  that  the  correlation  cost  model  and  the  causation  model 
are  interrelated  by  Equation  3. 

w = B x n (3) 

where  B is  the  weight  of  a single  bolt, 

w is  the  total  weight  of  all  of  the  bolts  that  go  into  the  assembly. 

Thus  any  design  or  sensitivity  analysis  performed  on  Equation  1,  the 
correlation  cost  model,  will  lead  to  the  correct  results  if  Equation  3 is  not 
violated.  For  example,  an  analyst  would  be  correct  in  predicting  that  a cost 
reduction  would  occur  if  he  reduced  the  weight  of  the  fasteners  used  by  using 
less  fasteners.  He  would  be  incorrect  if  he  predicted  a cost  reduction  would 
occur  if  he  reduced  the  weight  of  the  fasteners  by  substituting  aluminum  for 
steel  bolts  while  keeping  the  number  of  bolts  constant.  The  reason  that  a 
substitution  of  aluminum  for  steel  bolts  would  not  reduce  the  cost,  is  because 
the  underlying  relationship  between  the  number  of  bolts  and  the  weight  of  the 
fasteners  (Equation  3) , which  is  the  reason  for  the  good  cost  weight  relationship 
of  the  correlation  model,  has  been  violated. 

In  mathematical  terms  both  a causation  and  a correlation  cost  model  have 
the  following  properties. 


Cost  = f (characteristics)  (4) 

But  only  a causation  model  can  be  manipulated  as  Equation  5. 

Characteristics  = f”^  (cost)  (5) 

The  problem  of  determining  whether  a cost  model  is  a correlation  or  a 
causation  model  is,  except  for  the  trivially  simple  type  of  problem  illustrated  here, 
very  difficult  since  all  causation  models  can  be  transformed  into  correlation  models. 
There  exist  no  statistical  tests  to  determine  whether  a model  is  a causation  model  or 
a correlation  model. 

The  types  of  explanatory  variables  used  in  the  cost  model  generally  will 
give  a good  guide  as  to  whether  a model  is  a correlation  model  or  a causation  model. 

For  example,  weight  as  an  explanatory  variable  in  a cost  model  where  the  material  cost 
did  not  dominate,  would  be  a good  indication  that  the  cost  model  was  a correlation  model 

If  the  model  is  a correlation  model  and  the  analyst  performs  a sensitivity 
analysis,  he  runs  the  risk  of  violating  the  unknown  underlying  relationships  between 

* Morris  Zusman,  "Use  of  Cost  Models  in  Sensitivity  Analysis  and  as  a Design  Aid," 
Institute  of  Defense  Analyses,  N-587(R),  September  1968.  In  this  discussion,  the 
term  correlation  is  used  figuratively  in  the  sense  that  it  is  statistically  significant 
in  explaining  the  amount  of  variance  rather  than  in  the  sense  that  both  the  dependent 
and  independent  variables  are  random. 


Che  correlation  and  causation  models.  If  these  underlying  relationships  are  violated 
the  sensitivity  analysis  will  be  erroneous. 

This  example  illustrates  that  regression  analysis  is  an  aid  to,  and  not  a substitute 
for,  experience  and  understanding. 

The  bibliography  presents  many  references  on  cost  estimating  relationships  (CER's) 
derived  by  linear  regression.  Some  of  these  are  almost  as  simple  as  the  cost  to  weight 
relationship,  but  hopefully  more  useful.  Some  require  the  use  of  a computer  program  to 
handle  the  large  number  of  variables.  The  next  section  will  look  at  the  results  of 
some  regression  models  relating  acquisition  costs  and  reliability. 

Improving  reliability  is  a universally  recognized  method  for  reducing  O&M  costs. 
However,  improving  reliability  requires  an  investment  during  acquisition.  To  properly 
trade-off  reliability  against  LCC  one  must  consider  both  aspects.  For  this  reason,  the 
Rome  Air  Development  Center  (RADC) , initiated  a program  for  determining  the  relationship 
of  reliability  to  acquisition  costs,  and  in  the  course  of  that  program  produced  two 
models  concerned  with  development  costs  and  one  concerned  with  production  cost.  The 
first  two  are  based  on  linear  regression  and  provide  examples  of  its  application  'to 
cost  modeling. 

RELIABILITY  ACQUISITION  COST  STUDY:  RADC- TR- 77- 334 , "Reliability  Acquisition  Cost  Study" 
details  the  efforts  performed  by  "the  General  Electric  Aerospace  Electronic  Systems 
Department,  Utica  NY  on  contract  to  RADC  to  provide  models  relating  development  costs 
to  reliability. 

The  objectives  of  this  study  were  to  develop  relationships  capable  of  determining 
and  predicting  the  costs  attributable  to  reliability  during  the  development  phase  of 
electronic  equipment  acquisition.  Further,  basic  relationships  were  to  be  developed 
equating  reliability  increments  to  increments  in  development  cost.  Specifically,  the 
key  objectives  identified  at  the  outset  of  the  study  were  to: 

. Develop  a relationship  between  the  equipment  reliability  and  the  total 
reliability  development  cost. 

. Develop  a relationship  between  reliability  element  costs  and  the  equipment 
reliability. 

The  study  dealt  with  three  reliability  elements  as  they  relate  to  ten  equipments 
from  two  manufacturers.  These  elements  are: 

. Reliability  Design  Program  - including  prediction,  failure  mode  and  effects 
analysis  (FMEA),  and  design  reviews. 

. Reliability  Parts  Program  - including  parts  screening  specification,  parts 
standardization  and  control,  and  vendor  control. 

. Reliability  Testing  Program  - including  evaluation  testing,  equipment 
environmental  screening,  and  reliability  demonstration  testing. 

The  approach  to  the  study  was  to  hypothesize  linear  models  (linear  in  coefficients 
but  not  necessarily  linear  in  the  variables)  for  developing  relationships  between 
equipment  reliability  and  reliability  cost;  collect  reliability  cost  data,  reliability 
data  (based  on  failure  rate  history)  and  normalization  data  on  the  various  equipments; 
correlate  the  data;  synthesize  the  data  to  the  models  and  iterate  the  models  to  satis- 
faction. All  of  the  data  analyses  were  performed  using  time-share  computer  programs 
developed  by  the  Information  Services  Business  Department  of  the  General  Electric  Com- 
pany in  cooperation  with  the  General  Electric  Corporate  Research  and  Development  Center. 

Multiple  regression  analyses  were  performed  to  develop  prediction  equations  for: 

. Total  reliability  cost  as  a function  of  resultant  equipment  MTBF  and 
quantity  of  parts. 

. Reliability  prediction  cost  as  a function  of  quantity  of  parts. 

. Reliability  design  review  cost  as  a function  of  quantity  of  parts. 

. Reliability  failure  modes  and  effects  analysis  cost  as  a function  of  the 
prediction  cost. 

. Reliability  design  cost  as  a function  of  quantity  of  parts. 

. Resultant  equipment  MTBF  as  a function  of  reliability  parts  program  cost, 
reliability  test  program  cost  and  quantity  of  parts. 

It  should  be  pointed  out  that  a formal  failure  modes  and  effects  analysis  effort 
was  performed  for  only  three  of  the  equipments  in  the  data  base.  Therefore,  the  pre- 
diction relationship  for  failure  modes  and  effects  analysis  cost,  being  developed  from 
only  three  pieces  of  data,  is  statistically  weaker  than  the  others.  Also,  since  there 
were  only  three  equipments  with  FMEA,  these  equipments  were  excluded  from  the  develop- 


ment  of  the  prediction  equation  for  the  reliability  design  cost. 
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The  prediction  equations  are: 

Total  Reliability  Cost 
CT  = 1.804  eR0.370Np0.684 

Reliability  Prediction  Cost 
CpR  = 119.16  + 0 . 096Np 

Reliability  Design  Review  Cost 
CDR  = 92.479  + 0 . 022Np 


Reliability  Failure  Modes  and  Effects  Analysis  Cost 
CFM  = °'80  CPR 

Reliability  Design  Cost  (by  definition) 

Cd  = CPR  + CDR  + CFM 

Reliability  Design  Cost  (excluding  FMEA) 

C,  = 242.2  + 0.121N 
d P 


Reliability  Parts  Program  Cost  plus  Reliability  Test  Program  Cost 


Resultant  Equipment  MTBF 
5. 360Cp1-422c. °-639 

9 = £ £ 

K 1.372 

N 

P 


(1) 

(2) 

(3) 

(4) 

(5) 

(5a) 

(6) 


(7) 


A summary  of  the  key  statistics  for  these  equations  is  shown  in  Table  5 for 
equations  (1),  (2),  (3),  (4),  (5a)  and  (7).  Equations  (5)  and  (6)  are  not  included  in 
the  summary  since  they  were  generated  by  definition  instead  of  by  performing  regression 
analyses . 

These  equations  allow  prediction  of  reliability  costs  and  resultant  equipment  MTBF's 
but  do  not  address  the  question  of  incremental  gain  in  equipment  MTBF  as  a result  of 
each  of  the  reliability  elements.  Consequently,  they  have  been  termed  gross  prediction 
equations . 

The  statistics  provided  through  the  regression  analyses  give  an  indication  of  the 
accuracy  of  the  equations.  As  can  be  seen  from  Table  3 the  index  of  determination,  R" 
indicates  that  the  prediction  equations  account  for  more  than  50  percent  of  the  initial 
variation.  Also,  from  Table  3,  a comparison  of  the  standard  error  of  estimate,  Se,  and 
the  standard  deviation,  Sy,  indicates  that  the  dependent  variable  can  be  predicted  more 
accurately  using  the  equation  than  by  using  the  mean  of  the  dependent  variable. 

Also,  the  adequacy  of  these  prediction  equations  can  be  assessed  by  comparing  the 
values  predicted  by  each  equation  with  the  actual  data  values  that  were  used  to  develop 
the  relationships  (see  Table  4) . 

In  addition  to  percent  difference  between  predicted  and  actual  values , approximate 
confidence  limits  about  the  actual  average  value  can  be  computed  through  the  use  of 
appropriate  factors.  Such  factors  are  shown  in  Table  5,  for  each  of  the  prediction 
equations  (for  60,  75  and  95%  confidence  limits).  The  confidence  limits  are  based  on 
the  assumption  that  the  independent  variable  values  for  each  prediction  are  equal  to  the 
average  values  of  the  appropriate  variable  in  the  sample. 
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As  can  be  seen  from  Table  4,  the  largest  deviation  for  the  predicted  values  was  the 
reliability  prediction  cost  for  radar  E which  was  high  by  a factor  of  almost  four.  Also 
the  design  is  high  by  a factor  of  three.  This  results  from  the  fact  that  the  actual  re- 
liability prediction  and  design  costs  for  this  equipment  are  low.  The  reason  they  are 
low  is  attributed  to  the  fact  that  this  radar  was  developed  in  parallel  with  the  digital 
portion  of  the  same  system.  This  is  digital  equipment  F in  the  data  base.  It  appears 
that  the  radar  benefitted  from  this  parallel  development,  which  results  in  a lower  cost 
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TABLE  3 

GROSS  PREDICTION  EQUATIONS  AND  STATISTICS  SUMMARY 


Equation 


= 1.80  eR0-37  Np0-68 


= 119.16  +• 0.096  N„ 


= 92.48  + 0.02  Np 


0.80  CpH 

242.2  4 0.121  N 


! 

R2 

0 

.957 

0 

.488 

0, 

.576 

0. 

.992 

0. 

,876 

0. 

801 

0.180 

* 0.759 

679.29 

895.54 

132.37 

191.70 

160.77 

1513.29 

286.53 

740.34 

. 

0.879 

* 0.479 

* These  result  from  log  linear  equations. 


TABLE  4 

COMPARISON  BETWEEN  PREDICTED  COSTS  AND  ACTUAL  COSTS  WITH 
PERCENT  DIFFERENCE 


7,636.8 

10,264.0 


1, 804.8 
3,  165 
-67  .02 


1,806.6  714.7 


11,621.7  2,664.8  2,817.7 
11,947.0  2,532.0  2,705.0 
-2.8  4.98  0.805 


1,781.4 
2,076.0 
3 


411.8  367.7 

517.0  544 

•27.98  -47.91 


2,  529.2 

2.531.2 
-0.07 

4.929.3 
6,329.5 
-28.40 


16,631. 
17, 119. 
-2.93 

1,  121.2 

1,350.0 

-20.4 


1,476.6 

1,496.0 

-1.31 

1,  537.4 
1,496.0 
2.69 

6,  160.2 

8,708 

-42.33 


1,530.4  522.6 

1,087.0  658.0 

28.97  -25.9 

1,592.6  560.7 

1,087.0  658.0 

31.75  -17.36 


2,080.6 

893.0 

60.08 


1,404.8  757.9 
1,427.0  713.0 
2.5  5.93 


2,260.3 

2.670.0 
-18.12 

2.340.0 

2.670.0 
-14.10 


5,849.1  2,402.7  6,346.8  5,612.3  1,634.0  9,361.4  2,092.512,299.8 

6,632.0  2,318.0  5,883.0  4,456.2  1,932.0  9,277.0  1, 856.8  | 2, 088 .0 

-13.38  3.52  7.31  20.59  -18.24  0.90  11,26 


TABLE  5 

CONFIDENCE  LIMIT  APPROXIMATIONS 


4 1 1 


Multiply  the  values  of  Cy  and  0R  by  the  following  factors  for  the  Confidence  Limit 
indicated. 


PREDICTION 

VARIABLE 

M 

0.60 

M 

0.75 

M 



1 

LOWER 

UPPER 

LOWER 

UPPER 

LOWER 

UPPER 

C 

T 

.05 

1 .05 

.93 

1 .07 

.87 

1.14 

0 

R 

1 

->>1 

CO 

1 . 28 

t 

1 — 

-1 

1 



1 .41 

1 -52 

1 .9 

Add  (subtract)  from  the  values  of  CpR,  CpR,  C^  the  indicated  values. 


PREDICTION 

VARIABLE 

1 M 

0.60 

M 

Q..75 

M 

Q QA  . _ 

LOWER 

UPPER 

LOWER 

UPPER 

LOWER 

UPPER 

C 

PR 

-191 

+191 

-263 

+263 

-495 

+495 

CDR 

- 37 

+ 37 

- 51 

+ 51 

- 96 

+ 96 

Cd’ 

' - 81 

+ 81 

-111 

+ 111 

-208 

+208 

in  prediction  and  design  due  to  commonality  of  tasks.  If  equipment  E is  excluded  from 
the  reliability  prediction  analysis,  the  index  of  determination,  R2,  increases  from 
0.488  to  0.812  and  the  standard  error  of  estimate  decreases  from  679.29  to  430.05. 

All  of  the  other  prediction  quations  yield  predicted  values  that  are  from  0.5  to  2 
times  the  actual  values.  It  is  felt  that  this  is  acceptable  for  initial  estimates 
based  on  the  limited  data  base  available  coupled  with  general  equipment  development  un- 
certainty. 

Equation  (1)  is  used  to  predict  the  total  reliability  program  cost,  C^.  0R  as  used 
in  the  equation  is  the  final  equipment  MTBF  requirement  in  hours  converted  to  a 60  per- 
cent lower  confidence  level  and  Np  is  an  estimate  of  the  number  of  electrical  components 
that  the  equipment  will  contain  (if  the  equipment  is  a modified  design,  only  the  new 
components  are  used).  Putting  this  information  into  the  equation,  one  will  obtain  a 
prediction  in  terms  of  a cost  normalized  to  a new  design,  airborne,  analog  configuration. 
If  the  equipment  is  expected  to  be  predominantly  analog,  this  normalized  cost,  Cj,  will 
be  the  actual  predicted  total  reliability  program  cost  in  mandays. 

Equations  (2)  and  (3)  are  used  to  estimate  the  reliability  prediction  cost,  CpR,  and 
the  reliability  design  review  cost,  CpR,  respectively  using  the  number  of  electrical 
parts,  Np . The  intercepts  of  these  equations  represent  the  fixed  cost  portion  of  these 
reliability  element  costs. 

Equation  (4)  gives  a prediction  of  the  reliability  failure  modes  and  effects  analy- 
sis cost,  Cp^j,  if  it  is  required  on  the  equipment,  in  terms  of  the  effort  expended  in 
reliability  prediction  CpR.  This  predicted  cost,  Cppj,  is  also  normalized  as  described 
above.  Equation  (4)  was  developed  using  the  actual  prediction  data  on  the  10  equipments 
of  the  study,  and  in  the  application  of  the  equations,  one  uses  the  predicted  reliability 
prediction  cost  from  equation  (2)  as  the  value  for  CpR  in  equation  (4) . 

Equation  (5)  yields  a normalized  predicted  cost,  Cj,  for  the  reliability  design  pro- 
gram. This  equation  was  defined  using  the  actual  data  collected  and  in  application 
yields  a normalized  predicted  reliability  design  program  cost  by  adding  up  the  pre- 
dicted values  obtained  from  equations  (2),  (3)  and  (4).  If  the  equipment  for  which 
costs  are  being  predicted  does  not  have  a failure  modes  and  effects  analysis  requirement, 
then  equation  (4)  is  not  used  and  the  last  term  of  equation  (5)  is  zero.  Also,  if  there 
is  no  FMEA,  equation  (5a)  can  be  used  to  predict  the  design  cost. 

Equation  (6)  yields  a normalized  predicted  cost  for  the  combined  reliability  parts 
and  reliability  test  programs.  Again,  since  this  equation  was  developed  from  the 
actual  data,  the  output  from  equation  (5)  subtracted  from  the  output  of  equation  (1) 
will  yield  a normalized  predicted  cost  for  the  reliability  parts  and  test  programs  com- 
bined. To  obtain  the  actual  cost  for  digital  equipments,  the  parts  portion  must  be 
divided  by  an  appropriate  "K"  factor. 

Equation  (7)  is  used  to  predict  the  resultant  equipment  MTBF  (at  lower  60  percent 
confidence  level)  in  terms  of  the  normalized  reliability  parts  program  cost,  the  nor- 
malized reliability  test  program  cost  and  the  quantity  of  electrical  parts.  The  pro- 
cedure for  using  this  equation  is  to  first  take  the  outputs  from  the  other  equations 


L 


i 


* 
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to  determine  the  combined  cost  for  the  reliability  parts  and  test  program.  Next,  use 
equation  (7)  to  determine  the  allocation  of  this  cost  between  the  parts  program  and  test 
program.  This  allocation  reflects  the  average  of  the  actual  allocation  of  the  ten 
equipments  in  the  data  base  and  not  the  most  optimum  allocation. 

The  study  included  more  details,  such  as  the  modification  of  the  formulas  to  cover 
digita1  components  and  the  optimization  of  costs  between  the  three  reliability  elements, 
but  the  discussion  above  should  suffice  to  illustrate  the  use  of  regression  analysis  to 
develop  the  relationship  between  reliability  and  development  cost. 

RELIABILITY  VS  ACQUISITION  COST  (II)  : The  General  Electric  study  was  performed  on 
avionic  equipment.  Following  this,  RADC  awarded  a contract  to  the  Hughes  Aircraft  Com- 
pany, Ground  Systems  Group,  Fullerton  CA  for  a similar  study  on  ground  and  shipboard 
electronic  equipment.  The  final  report,  "RADC-TR-75-270  - Reliability  Acquisition  Cost 
Study  (II)'1,  provides  more  insight  into  the  variations  and  complexities  of  regression 
analysis,  and  excerpts  from  this  report  follow. 

SUMMARY  OF  DATA : The  data  for  the  systems  investigated  for  the  study  comprise  two 
categories.  The  first  category  contains  system  characterization  and  reliability  data. 
This  data  gathered  on  each  system  expresses  the  type  and  function  of  the  system,  the 
reliability  values  in  terms  of  mean  time  between  failure  (MTBF)  for  the  contractual 

ted  (0prpf)).  and  the  demonstrated  (9jjelno)  re- 
ty  as  defined  by  the  number  of  total  parts  (N) . 
The  number  of  parts  excludes  hardware  and  equipment  that  had  no  direct  effect  on  the 
system  reliability. 

The  number  of  parts  was  also  represented  by  the  total  number  of  system  analog  parts 
(N^)  and  the  total  number  of  system  digital  parts  (Np) . This  data  is  summarized  in  the 
System  Characterization  Table,  Table  6.  The  second  category  contains  the  reliability 
cost  data.  The  reliability  program  was  defined  by  determining  the  reliability  program 
phases  (see  Section  2.4.1)  and  grouping  the  reliability  program  costs  under  the  three 
program  phases.  The  reliability  program  phase  costs  are  the  reliability  design  phase 
costs  (Cj)) , the  reliability  parts  phase  costs  (Cp) , and  the  reliability  evaluation  phase 
costs  (Cg).  The  total  reliability  program  costs  (C^)  are  the  summation  of  the  three 
reliability  program  costs  and  all  cost  values  are  expressed  in  mandays . This  data  is 
summarized  in  the  Reliability  Program  Phase  Costs  Table,  Table  7. 

TABLE  6 

SYSTEM  CHARACTERIZATION  DATA 


specified  (9_  = 9spec.) , the  design  predic 
liability  values,  and  the  system  complexi 


Mean  Time  Between  Failures  (Hours) 

System  Number 

TVpe  or  Application 

Specified 

Predicted 

Demonstrated 

Total  Parts 

1 

Shipboard  ECM  System 

90 

98 

183 

42901 

2 

Shipboard  Target  Acquisition 
System 



125 

146 

81<2> 

16863 

3 

Shipboard  Radar  System  Solid 

State  Receiver 

500 

574 

300 (1* 

11313 

4 

Submarine  Fire  Control  Display 

290 

270 

851 

21683 

5 

Low  Frequency  Sonar  with  Towed 
Array  System 

182 

165 

264 

49243 

6 

Complex  Portable  Tracking  and 
Control  Center 

190 

210 

266 

36493 

7 

Artillery  Locating  Radar 

250 

216 

216 

18469 

8 

2400  Bit  Per  Second  Modem 

4000 

5721 

2005m 

1031 

9 

Tank  Fire  Control  ballistics 
Computer  System 

184 

369 

265 

1751 

10 

Satellite  Communications  System 

200 

217.5 

173 

17108 

(1)  Reliability  Demonstration  Test  terminated  at  minimum  acceptable  time  with  no  failures. 

(2)  Reliability  Demonstration  Test  conducted  by  U.  S.  N.  personnel. 


TABLE  7 

RELIABILITY  PROGRAM  PHASE  COSTS 


System  Number 

Design  Phase 

CD 

(man-days) 

Parts  Phase 
Cp 

(man-days) 

Evaluation  Phase 
CE 

(man-days) 

Total 

Ct 

(man-days) 

1 

214 

4962 

2498 

7674 

2 

244 

5301 

2196 

7741 

3 

202 

4093 

2249 

6544 

4 

207 

4467 

2464 

7138 

5 

237 

3580 

1348 

5165 

6 

204 

7233 

9262 

16699 

7 

170 

1612 

530 

2312 

8 

119 

3396 

1452 

4967 

9 

272 

1252 

928 

2452 

10 

148 

1449 

2110 

3707 

DEFINITIONS: 

Cp  - Reliability  design  phase  cost  (in  man-days) 

Cp  - Relative  reliability  design  phase  cost,  Cp  = Cp/CT 
Cg  - Reliability  evaluation  phase  cost  (in  man-days) 

I I 

Cg  - Relative  reliability  evaluation  phase  cost,  Cg  = Cg/Cp 
Cp  - Reliability  parts  phase  cost  (in  man-days) 

t I 

Cp  - Relative  reliability  parts  phase  cost,  Cp  = Cp/Cp 

Cp  - Total  cost  of  reliability  program  (in  man-days) 

Gp  - Reliability  gain  due  to  reliability  design  effort,  Gp  = Op/O^ 

Gg  - Reliability  gain  due  to  reliability  evaluation  effort,  Gp  = fig/ 

Gp  - Reliability  gain  due  to  reliability  parts  effort,  Gp  = 9p/0p 

GT  - Reliability  gain  due  to  total  reliability  program,  Gp  = GpC,pGg 

k- factor  - Adjustment  factor  for  environmental  applications 

N - Total  number  of  digital  and  analog  parts,  N = + Np 

Na  - Total  number  of  system  analog  parts 

Np  - Total  number  of  system  digital  parts 

NgA  - Number  of  system  parts  normalized  to  analog 

9p  - Post  design  MTBF  (in  hours) 

0 - Post  evaluation  MTBF  (in  hours) 

L 

9j  - Initial  system  MTBF  without  reliability  enhancement  (in  hours) 
9p  - Post  parts  MTBF  (in  hours) 

9pred  * Predicted  MTBF  (in  hours) 

" Specified  MTBF  (contractual)  (in  hours) 
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R 


100 


R.  E. 


1) 


Y 


i 


n 


2 V” 


where 

Y.  = the  ith  observed  value  of  a particular  function, 
o.  th 

Yj  = the  i calculated  value  of  a particular  function  (calculated  from  a model), 
n = number  of  observations. 


DATA  NORMALIZATION:  There  are  usually  a number  of  independent  variables  that  can  affect 
a given  dependent  variable.  In  this  study  we  will  particularly  be  investigating,  as 
dependent  variables,  reliability  and  cost.  The  important,  useful  independent  variables 
are  included  in  the  regression  models.  However,  there  are  several  (independent)  vari- 
ables that  may  affect  the  dependent  variables  which  we  do  not  particularly  want  to  include. 
These  will  be  "normalized”  out  or  excluded  from  the  regression  model.  These  variables  are: 

. Complexity 
. Design  differences 
. Environmental  differences 
. Time  differences 
. Non-relevant  cost  differences 

In  this  study  complexity  was  measured  by  parts  count.  The  variable  N (total  parts 
count)  and  (N*,  N^)  were  included  in  the  regression  models.  To  be  "on  the  safe  side" 
the  digital  parts  count  (Np)  on  each  system  was  converted  to  equivalent  analog  parts 
count  and  a new  independent  variable,  the  total  number  of  equivalent  analog  parts 

was  included  in  the  regression  models.  The  conversion  factor  used  was  a constant 
2 x Np  = (two  times  the  number  of  digital  parts  equals  their  equivalent  number  of 
analog  parts) . 

The  design  effects  are  considered  to  be  negligible  because  all  ten  (10)  systems 
in  the  data  base  were  designed  by  the  Hughes  Aircraft  Company,  nine  of  these  systems 
were  designed  at  Hughes-Fullerton. 

The  environmental  differences  were  removed  from  the  MTBF's  by  normalization.  This 
normalization  was  important  only  when  predicted  or  specified  MTBF  was  used  as  a de- 
pendent variable.  Also,  when  ratios  (say  9Spec^®pred^  were  used  as  dependent  variables 
the  normalization  factor  was  not  needed  because  it  cancelled  out.  The  basis  for  the 
normalization  is  given  in  Table  3.1.1.  The  k-factors  were  developed  using  the  fixed 
ground  environment  as  the  normal  (k  factor  = 1)  case.  The  k-factors  are  composites  of 
the  various  environmental  factors  given  in  MIL-HDBK-217B  for  the  various  parts.  The 
weights  assigned  were  based  on  rough  estimates  of  part  distribution.  It  turned  out 
that  the  shipboard  and  ground  mobile  factors  were  identical. 

The  ten  (10)  systems  forming  the  data  base  are  of  relatively  recent  vintage  so  that 
time  effects  are  considered  negligible.  Finally,  to  remove  non-relevant  cost  dif- 
ferences (e.g.,  the  changing  value  of  the  dollar)  all  costs  are  measured  in  man-days. 


TABLE  8 

k- FACTORS  FOR  ENVIRONMENTAL  NORMALIZATION 


Sys . # 

Environment 

e’ 

spec 

0’  . 
pred 

i 

Shipboard 

90 

98 

2 

Shipboard 

125 

146 

3 

Shipboard 

500 

574 

4 

Shipboard 

290 

270 

5 

Shipboard 

182 

165 

6 

Ground, 

Fixed 

190 

210 

7 

Ground, 

Fixed 

250 

216 

8 

Ground, 

Fixed 

4000 

5721 

9 

Ground, 

Mobile 

184 

369 

10 

Ground, 

Mobile 

200 

217.5 

adjusted 
( Ox  k -factor) 


k-factor 


THE  REGRESSION  VARIABLES:  The  various  sets  of  dependent  and  independent  variables,  60 
in  all,  are  shown  in  Table  9.  Each  set  was  run  on  each  of  the  five  model  types  dis- 
cussed in  the  next  section  with  the  exception  that  in  a very  few  cases  where  there 
existed  a large  number  of  independent  variables  (e.g.  3)  and  a small  number  of  data  sets 
(as  in  the  gain  analysis)  the  second  degree  model  with  cross-product  terms  could  not 
be  run  because  the  degrees-of- freedom  were  to  small.  Not  all  of  the  independent  vari- 
able sets  provided  good  predictions  for  the  various  dependent  variables  so  the  list  in 
Table  9 gives  all  those  sets  tried,  not  the  sets  that  were  good  fits. 

For  all  sets  the  dependent  variable  is  always  cost.  The  independent  variable(s)  are 
those  that  would  normally  be  available,  at  least  in  estimated  form,  early  enough  to  be 
of  use  in  predicting  costs. 

DESCRIPTION  OF  THE  MODELS  AND  MEASURES  OF  FIT:  Each  of  the  following  five  models  was 
fitted  to  each  of  the  variable  sets  whicK  are  seen  depicted  in  Tame  y.  it  was  not 
expected,  nor  even  desired,  that  all  models  would  fit  all  the  variable  sets.  Nor  was 
it  desired  that  there  be  at  least  one  good  model  fit  on  every  variable  set.  AH  that 
is  needed  is  that  there  be  at  least  one  good  model  for  each  dependent  variable  of 
interest.  For  example,  in  Table  9,  the  variable  sets  include  15  with  C^  as  the  de- 
pendent variable.  As  a worst  case  we  only  need  one  model  to  be  a good  fit  to  one  of 
these  15  variable  sets  with  Cf  as  the  dependent  variable.  The  five  models  used  are 
(t  represents  the  random  error  term) : 


Linear 


Ln-Linear 


Exponential 


Second  Degree 


Y = aQ  + a^  + . . . + anXT1  + e 

In  Y = In  Sq  + a-^lnX^  + . . . 4 anlnXn  4 e 


Y = eCa0  + alXl  + •••  + anXn  + r) 

Y = aQ  4 auXl  4 a12xj  4 ...  4 a^  4 a^  4 t 


Second  Degree  with  Cross  Products  (example,  for  n = 2) 

Y - a0  + aHXi  + a12XJ  + a21X2  + a22X2  + a(12)XlX2  + E 


TABLE  9 

THE  REGRESSION  VARIABLES  (Cont'd) 
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Selection  No 

Dependent  Variable 

Independent  Variable (s) 

1-37 

CD 

6 , N 

spec’ 

1-38 

CD 

®spec’  NEA 

1-39 

CD 

e , na. 

•spec*  A*  D 

1-40 

CD 

e ,,  n 

pred’ 

1-41 

CD 

0pred’  NEA 

1-42 

CD 

6pred’  NA’  ND 

1-43 

CP 

^pec^pred’  N 

1-44 

CP 

^spec^pred’  NEA 

1-45 

CP 

© /e  ,,  N.,  N_. 
spec  pred  A D 

1-46 

CP 

6spec’  N 

1-47 

CP 

6spec’  NEA 

1-48 

CP 

espec’  NA’  ND 

1-49 

CP 

®pred’  N 

1-50 

CP 

®pred’  NEA 

1-51 

CP 

°pred’  NA’  ND 

1-52 

CE 

^spec^pred’  ^ 

1-53 

CE 

®spec,/,®pred’  “^EA 

1-54 

CE 

^spec^pred’  NA'  ND 

1-55 

CE 

e , N 

spec 

1-56 

CE 

a N 

^spec*  EA 

1-57 

CE 

e , n,,  n_ 

spec  A’  D 

1-58 

CE 

6 j,  N 
pred’ 

1-59 

CE 

®pred’  NEA 

1-60 

CE 

0 .,  N.,  N_ 

pred’  A’  D 

i 


' 


l 


□ 

. 

J 

i 


■3 


r 


GOODNESS  OF  FIT  OF  MODELS:  For  applications  of  the  results  of  this  study  only  one  of 
the  above  models  is  needed  for  any  particular  set  of  variables  (Y,  X^  , ....  X_)  . Also, 
it  is  of  interest  to  see,  over  a variety  of  situations,  whether  one  particular  model 
is  invariably,  or  even  frequently,  the  best  fitting  model. 

Because  of  the  absurdity  of  the  assumption  that  any  particular  data  set  (of  dependent 
and  some  independent  variables)  is  a random  sample  from  a multivariate  normal  distribu- 
tion, the  usual  measures  of  goodness  of  fit  (F  test,  t test  and  Correlation)  have  been 
abandoned.  The  two  measures  of  goouness  of  fit  which  we  have  selected  are  R and  R.E.. 
The  formal  definitions  of  these  quantities  were  given  in  a previous  section.  In  words, 

R measures  the  average  (arithmetic  mean)  absolute  value  of  the  relative  (to  the  observed 
values)  deviation  of  the  observed  and  calculated  (from  the  model)  values  of  the  de- 
pendent variable.  R.E.  measures  the  fraction  of  the  unexplained  variation  to  the  total 
variation.  The  smaller  the  values  of  R and  R.E.  for  a particular  data  set  the  better 
the  fit.  The  ideal,  but  impossible,  situation  would  be  R - 0 and  R.E.  = 0. 

MULTIPLE  LINEAR  REGRESSION  METHODOLOGY:  Given  a linear  equation  in  two  variables, 

Y = a + BX  where  a is  the  Y intercept  and  S is  the  slope  of  the  line,  the  problem  of 
finding  the  "best  fit"  line  to  a given  set  of  N points  (xy,  vi ) , (xj,  l^),  •••  . 

(xm,  ypj)  is  to  determine  the  values  a and  b so  that  the  sum  of  the  squares  of  the 
difference  between  the  estimated  values  of  Y (given  by  y = a 4 bX)  and  the  observed 


4 IS 


values  of  Y is  a minimum.  This  is  the  least  squares  approach. 

The  constants  a and  b of  the  equation  Y = a + bX  are  solutions  of  two  linear  equa- 
tions called  normal  equations. 


aN  * b 


N N 

i=l  i=l 

N N N 

aZX + bSxf = SxiYi 

1=1  i=l  i=l 

The  constants  a and  b are  given  by 

N N N 

N£xivExiI>i 

i=l i=l  i=l 


b = 


N / N 


and 


a = 7 - bX 


where  X is  the  mean  of  the  X- values  and  Y is  the  mean  of  the  Y-values. 


In  the  multivariable  case  the  normal  equations  are  similar  to  the  linear  case.  The 
Y-values  are  the  dependent  variables,  i.e.,  in  the  analysis  of  the  data  the  dependent 
variables  can  be  reliability  phase  cost  or  total  cost,  and  the  X- values  are  the 
independent  variables,  i.e.,  number  of  system  parts,  reliability  phase  costs,  system 
MTPT  , depending  on  the  relationship  that  is  being  analyzed. 

New  software  was  created  to  calculate  the  deviation  (error)  from  the  regression 
analysis  data.  The  measures  of  "goodness  of  fit”  (R.E.  and  R)  were  determined  for  each 
selection  for  all  models  and  sorted  for  the  "best  fit"  by  selection  number. 

RESULTS  OF  DATA  ANALYSES:  The  variable  sets  as  previously  mentioned,  all  have  reli- 
abilitv  cost,  in  some  form  or  other,  as  the  dependent  variable.  The  distribution  of 
the  sixty  sets  (selections)  is 

SELECTION  NUMBERS  DEPENDENT  VARIABLE 


1-1  through  1-6 
1-25  through  1-33 
1-7  through  1-12 
1-34  through  1-42 
1-13  through  1-18 
1-43  through  1-51 
1-19  through  1-24 
1-52  through  1-60 


C 

C 

C 

C 

C 

C 

C 

C 


T 

T 

D 

D 

P 

P 

E 

E 


The  results  of  the  model  fits  are  given,  by  model,  in  Table  10.  In  the  following  sec- 
tions we  discuss  the  best  fit  results  for  each  individual  cost  category  and  total  costs. 


Results  for  total  cost,  Cy 


0.00819 


2 

spec 


+ 


1.89N.  - 
A 


CT  = -24,814  + 41.30  9 


spec 


0.000015N; 

A 


n * the  size  of  the  sample, 

x = the  specified  value  of  the  explanatory  variable  used  as  a basis  for 


U 


+0.40N-  + 0.0000082N2  - 0.0013N.  0 - 0.0032Nn  9 . (1) 

D D A spec  D spec 

C = -28,154  + 24.859  , - 0.003292  J + 2.34N.  - 0.000027N2 
T pred  pred  A A 

+0.76Nn  - 0 . 000061N2  - 0.0011N.9  , - 0.00064N^  9 (2) 

u DA  pred  D pred 


TABLE  10 

GOODNESS  OF  FIT  RESULTS  FOR  RELIABILITY  COST 


Linear 

Ln-] 

Linear 

Expon 

2nd 

Deg 

2nd  Deg  CP 

Selection 

R.E. 

R 

R.E. 

, R 

R.E. 

R 

R.E. 

R 

R.E. 

R 

i-i 

1.12 

56.27 

1.17 

46.93 

1.19 

48.06 

0.75 

35.04 

0.75 

35.04 

1-2 

1.08 

60.13 

1.09 

51.39 

1.18 

52.92 

1.01 

61.54 

1.01 

61.54 

1-3 

1.07 

59.44 

1.07 

51.63 

1.20 

56.42 

0.98 

58.23 

0.98 

58.23 

1-4 

0.88 

47.54 

0.95 

43.61 

0.95 

40.93 

0.81 

45.97 

0.81 

45.97 

1-5 

0.87 

48.22 

0.95 

43.35 

0.93 

41.07 

0.84 

47.46 

0.84 

47.46 

1-6 

0.  86 

48.  80 

0.93 

42.26 

0.92 

41.21 

0.75 

44.60 

0.61 

37.68 

1-7 

1.09 

19.31 

1.09 

19.74 

1.87 

27.64 

0.98 

19.  73 

0.98 

19.73 

1-8 

0.  76 

15.  51 

1.03 

19.47 

6.21 

45.  42 

0.69 

12.  43 

0.69 

12.  43 

1-9 

0.81 

15.07 

1.09 

19.38 

5.01 

39.28 

0.62 

12.99 

0.62 

12.99 

1-10 

0.94 

16.08 

1.05 

15.92 

0.95 

15.25 

0.85 

15.27 

0.85 

15.27 

1-11 

0.97 

16.86 

1.07 

16.48 

0.98 

16.23 

0.88 

16.40 

0.88 

16.40 

1-12 

0.90 

15.31 

0.61 

12.84 

0.92 

14.91 

0.76 

13.84 

0.65 

10.49 

1-13 

1.10 

60.27 

1.17 

53.80 

1.20 

55.33 

0.67 

40.02 

0.67 

40.02 

1-14 

1.11 

64.26 

1.17 

59.26 

1.22 

56.68 

1.05 

64.85 

1.05 

64.85 

1-15 

1.10 

64.68 

1.13 

59.47 

1.19 

58.38 

1.02 

62.70 

1.02 

62.70 

1-16 

0.73 

49.06 

0.87 

49.95 

0.80 

44.78 

0.72 

49.26 

0.72 

49.26 

1-17 

0.72 

49.31 

0.87 

49.84 

0.77 

43.82 

0.72 

49.24 

0.72 

49.24 

1-18 

0.  72 

49.53 

0.86 

49.25 

0.78 

44.22 

0.63 

43.35 

0.41 

32.60 

1-19 

1.12 

82.39 

1.20 

59.16 

1.18 

61.43 

0.88 

55.40 

0.88 

55.40 

1-20 

1.05 

92.76 

1.10 

62.94 

1.19 

69.12 

0.98 

100.00 

0.98 

100.00 

1-21 

1.05 

90.83 

1.11 

62.65 

1.21 

73.22 

0.97 

99.12 

0.97 

99.12 

1-22 

1.02 

73.05 

1.08 

54.48 

1.11 

55.63 

0.90 

87.73 

0.90 

87.73 

1-23 

1.01 

73.14 

1.08 

53.87 

1.09 

55.89 

0.93 

86.93 

0.93 

86.93 

1-24 

0.99 

74.18 

1.06 

52.33 

1.08 

55.36 

0.83 

91.90 

0.76 

76.93 

1-25 

0.85 

47.73 

0.86 

41.80 

1.33 

74.26 

0.50 

31.48 

0.45 

30.76 

1-26 

0.84 

48.47 

0.86 

41.38 

1.09 

60.97 

0.50 

31.26 

0.49 

30, 61 

1-27 

0.84 

48.59 

0.83 

41.36 

1.14 

64.49 

0.38 

27.84 

0.11 

15.51 

1-28 

0.88 

42.38 

1.04 

38.24 

1.03 

33.03 

0.69 

47.76 

0.46 

37.51 

1-29 

0.86 

41.73 

1.04 

37.68 

0.99 

32.88 

0.74 

48.02 

0.33 

32.92 

1-30 

0.85 

41.92 

1.02 

36.22 

0.98 

33.16 

0.61 

45.44 

0.00 

1.15 

TABLE  10 

GOODNESS  OF  FIT  RESULTS  FOR  RELIABILITY  COST  (Cont'd) 

Linear  Ln -Linear  Expon  2nd  Deg  2nd  Deg  CP 


Selection 

R.E. 

R 

R.E. 

R 

R.E. 

R 

R.E. 

R 

R.E. 

R 

1-31 

0.88 

42.73 

1.06 

35.52 

1.01 

34.37 

0.71 

44.46 

0.21 

17.85 

1-32 

0.86 

42.51 

1.05 

34.71 

0.98 

34.70 

0.77 

43.56 

0.15 

20.59 

1-33 

0.85 

43.01 

1.03 

32.76 

0.97 

34.88 

0.62 

42.88 

0.01 

6.05 

1-34 

0.71 

16.04 

0.54 

14.16 

14.55 

79.62 

0.31 

10.83 

0.31 

10.77 

1-35 

0.78 

16.68 

0.62 

14.99 

9.99 

64.78 

0.36 

11.95 

0.36 

11.93 

1-36 

0.60 

14.69 

0.11 

6.23 

28.67 

100.00 

0.16 

6.60 

0.15 

6.75 

1-37 

0.75 

15.32 

1.02 

17.91 

4.08 

36.57 

0.24 

8.43 

0.24 

8.24 

1-38 

0.76 

15.  51 

1.03 

18.51 

4.92 

39.57 

0.24 

8.18 

0.24 

8.31 

1-39 

0.66 

13.90 

0.45 

12.48 

5.67 

43.24 

0.24 

8.75 

0.11 

5.86 

1-40 

0.80 

14.96 

1.05 

16.77 

3.44 

32.68 

0.18 

6.75 

0.15 

6.00 

1-41 

0.81 

15.11 

1.07 

17.69 

4.11 

35.  59 

0.19 

7.19 

0.19 

7.15 

1-42 

0.73 

14.24 

0.54 

13.20 

4.22 

36.08 

0.22 

7.59 

0.11 

5.14 

1-43 

0.70 

48.97 

0.  SI 

49.18 

0.75 

45.19 

0.46 

32.93 

0.46 

32.62 

1-44 

0.70 

49.75 

0.81 

48.94 

0.44 

44.68 

0.49 

33.20 

0.49 

33. 16 

1-45 

0.70 

49.51 

0.79 

47.87 

0.74 

44.  89 

0.27 

27.95 

0.09 

11.65 

1-46 

0.64 

34.23 

0.96 

42.27 

0.88 

31.38 

0.53 

41.08 

0.37 

33.75 

1-47 

0.61 

33.94 

0.95 

41.99 

0.77 

30.01 

0.57 

38.75 

0.35 

30.48 

1-48 

0.60 

34.45 

0.92 

41.14 

0.77 

30.33 

0.40 

34.45 

0.03 

9.39 

1-49 

0.64 

36.28 

0.93 

40.41 

0.86 

33.15 

0.51 

36.70 

0.16 

18.31 

1-50 

0.62 

36.75 

0.93 

39.95 

0.77 

32.75 

0.56 

34.76 

0.20 

20.70 

1-51 

0.62 

37.09 

0.90 

38.70 

0.79 

32.54 

C,  38 

31.66 

0.01 

3.73 

1-52 

0.99 

71.79 

1.00 

54.93 

2.83 

100.00 

0.58 

53.80 

0.48 

56.85 

1-53 

0.98 

71.56 

0.99 

54.52 

2.39 

100.00 

0.50 

47.11 

0.46 

56.26 

1-54 

0.98 

72.37 

0.97 

52.24 

2.02 

100.00 

0.49 

45.99 

0.11 

28.70 

1-55 

1.01 

82.37 

1.12 

50.69 

1.14 

52.63 

0.79 

100.00 

0.54 

73.78 

1-56 

1.00 

81.02 

1.12 

49.54 

1.13 

52.30 

0.  82 

100.00 

0.30 

64.67 

1-57 

0.99 

80.43 

1.10 

47.45 

1.12 

51.77 

0.73 

100.00 

0.01 

13.69 

1-58 

1.01 

79.87 

1.15 

46.30 

1.13 

53.68 

0.85 

95.76 

0.31 

41.05 

1-59 

1.00 

78.79 

1.15 

44.67 

1.12 

53.57 

0.88 

96.  79 

0.15 

41.15 

1-60 

0.99 

79.09 

1.13 

43.53 

1.11 

53.18 

0.78 

94.26 

0.06 

27.96 

Results  for  Design  Cost,  CD 

The  best  fit  results  for  Cp  indicate  that  virtually  all  the  independent  variables  are 
good  predictors  of  CD>  Furthermore,  inspection  of  Table  10  indicates  that  all  of  the 
models  do  a reasonably  good  job  of  fitting.  One  example  of  such  a model  is: 

Cp  = 195.24  + .11  " .000032  .023  NA 

- 15  x 1CT8  (Na)2  - .094  ND  + 35  x 1CT7  (Np) 2 - 18  x 10‘6  0spEC  -Na 
+ 48  * 10'6  °SPEC  -ND 

k 


1 ' 


I 


Results  tor  part  cost,  Cp 


No  one  Independent  variable  does  well  as  a predictoi  for  C.j. . However,  again,  the 
sets  of  Independent  variables  0Sneo-  N^,  Np  (1-48)  and  0,,,.(,t|,  Np  (1-51)  are  pood  pie 
dlctors  of  Cp.  These  equation';  !ue  given  t'e tow . 

CD  - -4,133  + 15.020  „„„  - 0.00290-’  t I.  ION.  0.000010N; 

T spec  spec  A A 

- 1 . 07Nn  t 0.000  149N'’  - 0.00063N,  0 - 0.00044N-.  0 ( 1) 

0 D A spec  0 spec 


c 


T 


- 1 1 ,b0  1 t 10.080,  , - 

pt  eu 

t 0.'i.’Nn  - 0.000021Nj’ 


0.001 30"’  t 0.93N  - 0.00001  in; 

prod  A A 

0.0003  IN,  0 , - 0 . 00061N,,  0 

A pred  11  prod 


(4) 


Results  for  evaluation  cost,  C,. 


None  of  the  single  (independent)  variable  eases  provided  suitable  fits.  Again,  the 
independent  variables  8SpOC,  N^,  Np  and  0 proij,  N(^,  Np  provide  satisfactory  fits  for  the 
2nd  degree  cross  products  model.  These  equations  (selections  1-57  and  1-60)  are  given 
be l ow . 


■;  ■> 
c..  -17,117  l .’8.1  70  - 0 . 00560'  l 0.74N,  - 0.0000048N' 

b spec  spec  A a 

+1 . 78N_  - 0 . 00005  IN'  - 0.00067N.0  - 0.0031N.O 

D 11  A spec  I)  spec 

C,.  - 1 8 , 0 .’  7 t 1 5.770,,,  , , - 0 . 007  1 O'  l 1 ,49N.  - 0.00001  7N-’ 

fc  l,,l“  pred  A A 

t0.4lNn  - 0 . 000048N"’  - 0.00080N,  0 - 0.00014N.,  0 

■’  D A pred  1>  pred 


(51 


(6) 


ExampJ.es  (The  systems  used  in  the  examples  were  randomly  selected) 
l)  Predicting  total  cost  Cj : 

System  No.  6 has 


spec 


140 


"pred 


310 


N,  28,429 

A 

Nn  8,0b4 

C_  16.649. 


Using  equation  (!)  we  obtain 

CT  —24,814  t 41.30(140)  0.0081(190)'’  t 1.89(28,429) 

-0.000015(78.479)-’  t 0.40(8,064)  0.0000082(8,084)' 

-0.001  1(190)  (78,429)  - 0 . 0032(  14(1)  (8 , 084) 


- 1 6 , 466 . 5 3 

which  is  in  close  agreement  with  C.j.(ORS)  16.699.  Also,  equal  ion  (3)  leads  to 
CT  - -28,154  t 24.85(210)  - 0.0037(210)'’  t 2.34(28,429) 

-0.000027(28,429)-’  t 0.76(8.064)  - 0.000061  (8.064)' 

-0.001 1 (210) (28. 474)  - 0. 00064(210) (8,064) 


- 16,137.65. 

7)  Predicting  design  cost  C^ 


System  No. 

1 has 

0 

360 

spec 

0 

392 

pred 

28,097 


4-:: 


N = 14,804 
D 

CD  = 214‘ 

To  avoid  repetition  we  will  use  just  the  set  0 


N.,  Nn.  From  (1-39) 


CD  = 195.24  + 0.11(360)  - 0 . 000032 (360) 2 + 0.023(28,097)  - 0.00000015 
x(28 , 097) 2 - 0.094(14,804)  + 0 . 0000035 (14 , 804) 2 - 0.000018(360) 
x28 , 097  + 0.000048(360) (14,804) 

= 207.71 

This  agress  very  well  with  the  CD(0BS)  = 214 

3)  Predicting  part  cost  Cp: 

System  No.  3 has 
9spec  = 2.000 
Opred  = 2,296 
Na  =9,461 

Nd  = 1,852 

Cp  = 4,093. 

Equation  (3)  gives 

Cp  = -9,133  + 15.02(2,000)  - 0 . 0029 (2 , 000)  2 4-  1.10(9,461) 

-0. 000010(9, 461)2  - 1.07(1,852)  + 0 . 000049 ( 1 , 852) 2 
-0.00063(2,000) (9,461)  - 0 . 00044(2 , 000) ( 1 , 852) 

= 3,454.80. 

This  result  agrees  well  with  Cp(OBS)  = 4,093. 

4)  Predicting  evaluation  cost  CE : 

System  No.  4 has 

0 = 1,160 

spec 

9pred  = 1.080 

Na  = 19,567 

Nd  = 2,116 

C =2,464. 

E 

Using  Equation  (5) 

CE  = -17,137  4-  28.17(1,160)  - 0 . 0056  ( 1 . 160)  2 + 0.79(19,567) 

-0.0000048  x (19.567)2  + 1.78(2,116)  - 0 . 00005 1 (2 , 1 16) 2 
-0.00067(1, 160) (19, 567)  - 0.0031  x ( 1 , 160) (2 . 1 16) 

= 2,346.55. 

RELIABILITY  TRADE-OFFS  FOR  UNIT  PRODUCTION  COST:  We've  spent  a lot  of  rime  on  regression 
analysis  because  it  is  important  for  the  user  to  understand  its  difficulties  as  well  as 
its  advantages.  The  final  RADC  study  in  reliability  and  acquisition  cost  did  not  develop 
regression  equations  and  hence,  provides  a sample  of  an  alternate  means  of  cost  estimating 
in  acquisition. 


The  study  was  performed  for  RADC  by  the  Martin-Marietta  Corporation,  Orlando  FL. 

At  this  writing  (January  1979)  the  final  report,  "Reliability  Trade-Offs  for  Unit  Pro- 
duction Cost"  is  still  in  printing.  Excerpts  follow 

A model  that  relates  reliability  achievement  t o the  many  program  trade-offs  and 
their  resultant  cost  consequences  should  delineate  physical,  measurable  tasks  or  ac- 
complishments and  he  subdivided  along  lines  that  are  traditional  for  cost  collection  and 
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Below  is  a general  expression  for  electronic  equipment  reliability  that  reflects 
three  primary  influences:  development  engineering,  manufacturing,  and  field  use  and 
support. 


R = MTBF  (R  ) (R  ) (R  + R,  + R,) 
p g m s f d 


where 

R = mean  time  between  failures  at  any  given  time 


MTBF  = predicted  mean  time  between  failures  in  accordance  with  procedures 
" in  M1L-HDBK-217.  This  parameter  implicitly  accounts  for  the  quality 
of  purchased  parts  and  the  electrical  and  environmental  stress  and 
derating. 


reliability  growth.  This  factor  generally  varies  between  0.1  and  1.0 
and  relates  to  the  adequacy  of  the  test-analyze- f ix  program  during 
engineering  development.  A good  program  will  result  in  a value  of  R, 
equal  to  0.9  or  greater  after  about  10,000  hours. 


g 


manufacturing  influence.  This  factor  relates  to  the  assurance  that  any 
manufacturing  induced  defects  have  been  detected  and  removed. 


Rs  + Rf  + Rd  = post  deployment  logistic  support  influence.  Rs  is  a skill  factor 
of  operators  and  maintenance  personnel.  Rf  relates  to  the  adequacy  of 
facilities,  including  workshops,  special  test  equipment,  and  repair  parts. 
Rj  is  a measure  of  the  availability  and  quality  of  documentation,  including 
operating  and  overhaul  and  maintenance  manuals. 


The  basis  for  this  reliability  expression  and  the  elements  that  influence  it  are  shown 
pictorially  in  Figure  4.  It  is  significant  that  product  reliability  is  not  a constant 
value  but  varies  with  the  effects  of  these  influences.  The  basis  of  this  reliability 
expression  will  first  be  discussed  in  some  detail  and  then  related  to  impact  upon  unit 
production  cost. 


R = 


MTBF  (R  ) 
P g 


(R  ) 
m 


(R  + R ' R ) 
s f d 
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t 
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FIG.  4 - BREAKDOWN  OF  RELIABILITY  EQUATION  FOR  COST  TRADE-OFFS 


DESIGN  RELIABILITY  FACTOR,  MTBF  (R  ) : The  prediction  models  as  presented  in  MIL-HDRK-217 
are  developed  from  actual  f i elcrfaTlure  rate  experience  and  therefore,  are  the  logical 
reference  for  estimating  the  potential  reliability  of  any  given  equipment.  It  has  been 
shown  in  numerous  studies  that  individual  products  can  exhibit  widely  different  results 
when  compared  to  such  a prediction,  depending  primarily  upon  the  maturity  of  the  design, 
the  effectiveness  of  the  manufacturing  quality  control,  and  upon  operational  influences. 


General  Electric  has  published  studies  showing  that  reliability  demonstration  of 
radically  new  designs  will  tvpicallv  realize  approximately  10  percent  of  the  MIL-HDBK- 
217  prediction  value  initially,  improving  to  near  100  percent  according  to  a growth 
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curve  of  variable  slope  determined  by  the  effectiveness  of  reliability  engineering  dur- 
ing the  design  phase.  Accordingly,  the  first  two  factors  of  the  reliability  equation, 
MTBFp  and  R , account  for  the  influence  of  design  and  development  engineering. 

To  predict  the  reliability  (MTBF  ) of  a given  equipment  configuration,  the  component 
parts  are  identified  along  with  the  design  guidelines  for  environmental  and  electrical 
stress.  The  variables  are: 

. The  quality  of  parts  chosen,  such  as  established  reliability  (ER)  level 

L,  M,  P,  R,  S,  or  T for  passive  devices;  JAN,  JANTX,  or  JANTXV  for  discrete 
semiconductors;  and  MIL-M-38510,  class  C,  B,  or  A for  integrated  circuits. 

A typical  example  would  contain  a mixture  of  parts  and,  for  conceptual  design, 
may  have  to  be  grossly  estimated  as  provided  for  in  the  procedure  of  Section  3 
in  MIL- HDBK-2 17 . 

. The  ratio  of  applied  electrical  stress  to  the  device  maximum  rating. 

. The  environmental  exposure  limits  of  temperature,  vibration,  etc. 

With  these  variables  specified,  an  MTBFp  can  be  determined.  To  realize  the  potential  of 
this  prediction,  a test-analyze- f ix  program  is  essential  for  debugging  new  designs.  It 
generally  requires  repeated  cycles  of  operation  under  all  combinations  of  environmental 
stress,  with  positive  corrective  action  to  remove  design  defects. 

R is  related  to  the  intensity  and  duration  of  the  test-analyze- fix  program  accord- 
ing te  the  graph  shown  in  Figure  5.  When  a well  disciplined  reliability  program  is 
followed  with  quick  reaction  to  test  problems,  the  slope  of  the  curve  can  be  expected 
to  approximate  the  straight  line,  a = 0.5  as  plotted  on  log  graph  paper.  The  develop- 
ment of  this  relationship  is  covered  in  detail  in  "A  Learning  Curve  Approach  to  Reli- 
ability Modelling",  J.  T.  Duane,  IEEE  Transactions,  Aerospace,  Vol  2,  1964.  When 
combined  into  an  integrated  test  program  (not  dedicated  to  reliability  growth  testing) 
or  with  less  than  top  priority  given  to  failure  analysis  and  corrective  action  design, 
the  growth  rate  can  be  expected  to  follow  the  slopeof  a = 0.25  or  less.  Evaluation  of 
trade-offs  requires  a judgement  as  to  the  appropriate  value  of  a. 


FIG.  5 - RELIABILITY  GROWTH  FACTOR,  R(- . versus  Cumulative 

grow™  testing  hours 


MANUFACTURING  INFLUENCE  FACTOR , R : In  theory,  it  docs  not  matter  that  numerous  defects 
are  introduced  into  electronic  equipment  during  the  manufacturing  process,  sc'  long  as 
they  are  detected  and  corrected  prior  to  final  delivery  of  the  product.  This  position 
will  be  examined  in  much  more  detail  in  a laLer  discussion  of  cost  consequences,  but  for 
the  moment  it  is  submitted  that  for  reliability  achievement , it  is  sufficient  to  employ 
limited  environmental  testing  (LET)  to  remove  c.or  confirm  the  absence  of)  manufacturing 

(defects  (material  and  workmanship).  The  more  rigorous  this  test,  within  design  limits, 
and  the  more  cycles  of  exposure,  the  greater  the  probability  that  all  manufacturing 
defects  have  been  removed. 

Figure  6 is  a graphical  representation  of  the  empirically  derived  equation  for  re- 
moval of  infant  mortality  failures  in  electronic  equipment  bv  use  of  repeated  variation 
of  environmental  stress: 


h 


Li 


: 

* : 
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«m  = 1 ' d ' Ec>n 

where 

Ec  is  a measure  of  the  severity  of  the  test  environments  relative  to  the 
design  limits 

n is  the  number  of  cycles  of  environmental  exposure. 


FIG.  6 - LIMITED  ENVIRONMENTAL  TESTING  IMPACT 


This  expression  is  verified  by  data  from  an  extensive  study  by  R.  W.  Burrows  in 
which  results  of  environmental  screening  practices  of  26  electronic  equipment  manufact- 
urers were  summarized.  The  curves  for  three  representative  values  of  Ec  are  shown.  One 
is  for  the  case  of  combined  exposure  to  the  two  most  critical  environmental  elements, 
such  as  temperature  cycling  and  random  vibration,  with  levels  equal  to  the  design  limits, 
Ec  = 0.5.  Another  curve,  Ec  = 0.3,  is  representative  of  temperature  excursions  of  50°C 
and  sinusoidal  vibration  at  2.2g.  The  third  curve  shown,  for  Ec  = 0.1,  represents  a 
burn-in  screen  where  temperature  is  not  cycled.  To  apply  the  equation,  eight  hours  of 
burn-in  is  equivalent  to  one  cycle  i.e.,  for  a 48-hour  burn-in,  n = 6.  For  test  pro- 
files that  do  not  approximate  either  of  these  cases,  a value  of  Ec  may  be  estimated  bv 
considering  the  relative  severity  of  the  intended  test  to  those  described  above,  keeping 
in  mind  that  Ec  = 0.5  is  probably  an  upper  limit  for  any  practical  application. 

OPERATION  AND  SUPPORT  FACTOR,  R?  + Rf  + Rd  : The  final  factor,  R + Rf  + Rd , is  included 
in  the  model  for  completeness  in  accounting  for  total  life  eye lesreliabi 1 i tv . It  has  a 
negligible  impact  upon  unit  production  cost  and  therefore,  will  not  be  expanded  upon 
further.  For  quantitative  reliability  predictions,  this  factor  will  approach  unity  as 
field  training  and  familiarity  with  the  equipment  increases.  A judgement  can  be  made  of 
the  appropriate  value  for  each  analysis  based  upon  comparison  to  current  or  recent  ex- 
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L “ direct  labor 
0^  = labor  burden  rate 

G<*A  =>  general  and  administrative  burden  rate 
Fee  - profit  fee  rate. 

M and  L are  the  variables  that  may  be  impacted  by  reliability  considerations  MTBF  and 
Rn,  as  shown  qualitatively  in  Figure  2. 

MTBFp  COST  IMPACT:  To  determine  MTBFp,  it  is  necessary  to  specify  the  quantity  and 
quality  of  component  parts,  including  any  requirement  for  supplemental  screening/testing. 
If  complete  material  lists  exist  for  all  design  configurations  that  are  to  be  considered 
for  trade-off  study,  conventional  cost  estimating  techniques  are  used  to  estimate  the 
cost  impact  of  each  design. 


For  conceptual  design,  where  detailed  parts  and  materials  lists  do  not  exist  for  all 
design  options,  MTBFp  is  estimated  by  following  the  Section  3 method  of  MIL-HDBK-217 , 
and  the  related  material  cost  may  be  estimated  from  the  relationships  shown  in  Figure  7. 
Data  was  collected  from  several  projects  to  compile  Figure  7,  which  shows  considerable 
variation  and  even  some  overlap  in  relative  cost  between  the  quality  of  part  levels. 

Note  that  many  different  types  of  parts  are  represented  and  that  judgement  is  required 
to  select  the  most  appropriate  values. 

In  general,  more  exotic  and  complex  parts  will  require  a greater  differential  in 
price  when  quality  requirements  are  increased.  Conversely,  common  and  simple  parts  can 
be  tested  extensively  with  relatively  simple  automation,  resulting  in  very  little  in- 
crease in  price.  Set  up  charges,  which  must  be  prorated  over  each  production  lot,  be- 
come quite  significant  when  small  quantities  are  involved.  These  factors  were  reflected 
in  the  data,  and  should  be  considered  in  using  Figure  7 to  estimate  parts  costs. 


Commercial  JAN  TX  TXV  Ultra, 

ER-L,M,  ER-P.R,  ER-S.T,  Minuteman 
38510-C  38510- B 38510-A  Space,  etc. 


FIG.  7 - COST  OF  PURCHASED  PARTS  VERSUS  SPECIFIED  QUALITY 


MANUFACTURING  COST  IMPACT  OF  RELIABILITY:  The  cost  impact  of  R„  (manufacturing  influence 
upon  reliability)  is  much  more  complex.  Selection  of  a value  of  R^  for  the  reliability 
equation  implies  that  a finished  assembly,  after  completion  of  LET  will  have  a correspond 
ing  freedom  from  manufacturing  defects.  There  may  have  been  many  failures,  subsequent 
repairs,  retesting,  or  none.  Only  the  end  result  is  significant  for  reliability  achieve- 
ment. 


427 


However,  the  means  of  realizing  the  failure-free  finished  product  may  be  very  sig- 
nificant to  the  costs  incurred.  Generally,  there  are  three  philosophies  for  setting  up 
the  manufacturing  operation: 


1.  Carefully  control  each  step  in  the  manufacturing  process  so  that  no  defects 
are  introduced.  This  is  the  preventive  approach  and  incurs  cost  for  care- 
ful analysis  and  planning  of  all  manufacturing  operations.  If  successful, 
few  if  any  failures  will  occur  during  LET  and  little  or  no  cost  is  incurred 
in  troubleshooting,  repair,  and  retest. 

2.  Somewhat  the  opposite  of  the  above  is  to  build  up  all  assemblies  with 
minimal  control  and  plan  on  multiple  repetitions  of  the  LET  to  remove  the 
numerous  defects  until  subsequent  test  cycles  demonstrate  the  unit  to  be 
failure  free. 

3.  The  other  philosophy  is  to  study  the  economics  of  the  application  of  the 
first  two  approaches  and  choose  to  emphasize  one  or  the  other  on  an  in- 
dividual manufacturing  operation  basis. 

Approach  number  1 must  be  followed  where  the  cost  of  a specific  failure  is  extremely 
high,  for  instance,  when  human  lives  may  be  threatened.  It  is  generally  the  preferred 
approach  for  very  complex  equipment  that  is  in  high  volume  production.  Number  2 is 
most  attractive  for  very  low  volume  production  or  for  relatively  simple  assemblies  where 
troubleshooting  and  repair  are  very  easily  accomplished. 

Most  electronics  manufacturing  companies  who  deal  with  government  contracts  set  up 
their  manufacturing  systems  to  facilitate  both  high  volume  and  low  volume  production. 
Thus,  in  most  practical  situations,  approach  number  3 is  selected. 

In  general,  manufacturing  defects  are  avoided  by  careful  adherence  to  detailed 
written  instructions  for  each  operation  that  have  been  thoroughly  tested  and  proven  to  be 
effective.  Such  validated  instructions  apply  to  all  manufacturing  operations  including 
procurement  of  material,  in-house  material  handling,  all  fabrication  and  assembly  tasks, 
inspection,  test  and  package  for  shipment.  Equal  attention  is  required  for  housekeeping 
activities  such  as  maintenance  and  calibration  of  test  equipment,  tools,  gauges  and 
measurement  standards;  personnel  training  and  proficiency  monitoring;  and  vendor  selec- 
tion, surveillance,  rating,  and  control.  To  ensure  that  these  instructions  and  controls 
are  effective,  a positive  acting  closed  loop  failure  analysis  and  corrective  action 
system  is  required. 

For  detailed  analysis,  direct  labor,  L,  is  subdivided  into  four  categories: 


L — L 4"  L,_  4-  L 4"  I*. 

assy  test  sup  LET 

where 

L„  = all  fabrication,  assembly,  and  rework  labor, 
assy  1 

Ltest  = in-line  test,  inspection,  and  troubleshooting 

L = all  quality  assurance  and  production  engineering  including  failure 
” analysis  and  followup 

Llet  = LET  labor. 


Emphasis  upon  preparation  and  maintenance  of  instruction  sheets,  along  with  failure 
analysis  and  corrective  action  generally  requires  an  increase  in  support  labor,  L , 
and  results  in  less  troubleshooting  labor,  L.-est,  and  less  rework  labor,  La  . " 

This  suggests  a subset  trade  study  to  find  tne  minimum  labor  cost  to  achieve'  the  reli- 
ability level  (freedom  from  manufacturing  induced  defects)  implied  by  R^.  An  example 
that  illustrates  these  cost  interactions  is  included  in  a later  section. 


One  manufacturing  control  element  that  is  very  important  to  the  efficient  production 
of  modern  avionics  equipment  is  the  incoming  inspection  of  electronic  parts.  When  a 
defective  part  must  be  detected  and  replaced  after  assembly  onto  a printed  wiring  board, 
it  is  commonly  estimated  that  the  cost  is  10  times  that  of  detection  at  the  incoming 
inspection  point.  Further,  if  the  printed  wiring  board  containing  a defective  component 
part  is  installed  into  a higher  assembly,  the  cost  estimate  for  detection  and  repair 
increases  to  a factor  between  30  and  100. 


The  cost  impact  of  incoming  inspection  effectiveness  is  vividly  illustrated  by 
Table  11  from  a recent  study  on  the  detection  of  integrated  circuit  defects.  Data  was 
recorded  for  the  distribution  of  detected  failures  per  thousand  ICs  tested  at  various 
assembly  levels  and  for  three  separate  incoming  inspection  methods.  One  method  was  to 
inspect  a sample  of  30  percent  of  parts  received;  a second  method  was  to  conduct  100 
percent  functional  tests,  and  the  third  was  to  combine  100  percent  environmental  screen- 
ing in  addition  to  100  percent  functional  tests.  Using  the  first  method,  30  percent 
sample  testing,  3.2  defects  per  thousand  devices  were  detected  at  incoming  at  an  average 
cost  per  defect  of  $0.50,  14.5  defects  per  thousand  devices  at  the  board  level  at  a cost 
of  $10  each,  3.2  defects  at  the  system  level  at  an  average  cost  of  $30  per  defect,  and 
1.4  were  detected  in  the  field  at  a cost  of  $100  per  defect.  The  detection  cost  per 
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defect  was  proportionately  higher  for  the  other  incoming  inspection  methods,  $1  per  de- 
fect for  method  two,  and  $6  each  for  method  three,  but  the  overall  cost  of  removing 
integrated  circuit  defects  from  a system  was  $180  less  for  method  two  and  $200  less  for 
method  three. 


TABLE  11 
DATA  SUMMARY 


Type  of  Incoming 

Failures  per 

1000  Tested 

Inspection 

Incoming 

Board 
$ 10 

System 
$ 30 

Field 
$ 100 

30  percent  of  devices 
tested 

3.2  0 
$0.50 

14.5 

3.2 

1.4 

100  percent  of  devices 
tested 

15 

$1.00 

4.6 

2.3 

0.7 

100  percent  tested 
plus  screening 

21  0 
$6.00 

2.3 

0.5 

0.16 

figure  8 shows  cumulative  cost  per  system  for  the  three  methods.  Another  recent 
study  suggests  that  in  the  general  case  the  cost  per  defect  at  the  system  level  and  in 
the  field  could  be  much  higher,  which  makes  the  effectiveness  of  incoming  inspection  even 
more  significant  than  this  example  indicates. 


FIG.  8 - CUMULATIVE  COST  TO  REMOVE  INTEGRATED 
CIRCUIT  DEFECTS  VERSUS  INCOMING  TEST 
METHODS 


The  cost  of  conducting  incoming  inspection  is  frequently  accumulated  as  a material 
handling  overhead,  designated  as  Om  in  the  UPC  equation.  For  trade-offs  involving  the 
degree  of  incoming  inspection,  this  factor  should  be  adjusted  accordingly.  The  procedure 
is  to  estimate  the  differential  cost  of  the  inspection  alternative  and  to  add  (or  sub- 
tract) from  the  standard  cost  (usual  or  baseline  method  of  inspection).  For  the  previous 
example,  the  differential  cost  of  adding  100  percent  screening  is  $6  per  defect  times  21 
defects,  less  $1  per  defect  times  15  defects  for  each  100  devices  tested.  If  the  material 
cost,  M,  is  assumed  to  be  $2000  per  1000  devices,  then  an  increase  in  material  burden 
rate  can  be  computed  by  dividing  the  added  test  expense  by  the  material  cost. 
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ag  = new  test  cost  - old  test  cost 
m total  material  cost 


(6)  (21)  - (1)  (15) 


126  - 15  = 0.056  or  5.6  percent 


If  the  standard  inspection  method  is  to  conduct  100  percent  functional  testing  with- 
out screening,  at  a 15  percent  total  material  handling  burden  rate,  then  a burden  rate 
of  20.6  is  used  for  any  trade-off  alternative  in  which  100  percent  screening  at  incoming 
inspection  is  chosen.  Those  companies  who  do  not  establish  separate  material  burden 

rates  may  evaluate  UPC  trade-offs  by  setting  0m  = A0m,  as  calculated  above,  for  any  de- 

viation from  the  standard  incoming  inspection  method. 

LET  COST:  The  cost  of  environmental  testing  (LET)  takes  the  form  of  a fixed  setup  cost 
plus  incremental  costs  dependent  upon  the  duration  of  the  test.  From  the  reliability 
considerations,  the  value  (or  set  of  values)  of  dictates  the  number  of  test  cycles  to 
be  conducted.  The  LET  cost  therefore  may  be  represented  by  the  equation: 

LLET  = Lcyc  ^ + Ls 

where 

Llet  = cost  of  conducting  a specific  LET 

L„  = cost  of  one  cycle  of  LET  exclusive  of  setup 

eye 

n = number  of  test  cycles  corresponding  to  Rm 
Ls  = setup  costs 

Graphical  interpretation  is  provided  in  Figure  9.  L and  L are  both  directly  pro- 
portional to  the  complexity  of  the  unit  under  test,  to  the  nufiber  of  measurement  points 
to  be  monitored,  and  to  the  length  of  time  required  to  traverse  one  cycle,  and  inversely 
proportional  to  the  amount  of  automation  employed. 


Number  of  test  cycles  •s  n 


Lcyc  * C0STS  0f  0NE  CYCLE  ExCLUSIVE  0F  SETUP 


FIG.  9 - LIMITED  ENVIRONMENTAL  TESTING  COST  MODEL 


The  cost  estimating  procedure  is: 

1.  Given  the  specific  test  requirements,  obtain  an  estimate  oc  the  setup  costs. 

2.  Obtain  an  estimate  of  the  cost  of  one  test  cycle. 

3.  Determine  the  number  of  cycles  required  by  the  value  of  the  Rm. 
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EVALUATING  TRADE-OFFS:  From  the  reliability  equation, 

R = MTBF„  (R  ) (R  ) (R  + R + R,) 
p g m s f d 

it  has  been  shown  that  variations  in  MTBF  and  R^  alter  unit  production  cost.  From  the 
UPC  equation,  V 

UPC  = (M  (1  + 0m)  + L (1  + Oj^))  (1  + G&A  (1  + fee)) 

M,  0^,  and  L vary  with  the  reliability  requirements.  For  evaluation  of  trade-offs,  a 
matrix  is  suggested  of  R versus  UPC  as  follows.  If  there  are  i alternatives  of  MTBF 
and  j alternatives  of  Rm,  a UPC^;  can  be  calculated  for  each  R^;  according  to  the  re- 
lationships developed.  After  calculating  all  combinations,  theJ trade-off  evaluator  has 
two  alternatives;  choose  the  least  cost  combination  that  produces  an  acceptable  reli- 
ability, or  choose  the  most  reliability  available  at  a given  cost. 

Example: 

Given  two  choices  each  for  MTBFp  and  Rm, 

MTBFp^  = 100  hours  (use  all  JAN  devices) 

MTBFp2  = 500  hours  (all  TX  devices) 

R^l  = 0.975  (six  LET  cycles,  Ec  = 0.5) 

Rm2  = °-75  (two  LET  cycles,  Ec  = 0.5). 

Since  the  factors  Rg  (for  purpose  of  the  example)  and  RQ  + Rf  and  R^  do  not  significantly 
affect  UPC,  they  may  be  removed  from  the  analysis  by  assuming  them  equal  to  1. 

Then 

RU  = 100  (1)  (0.975)  (1)  = 97.5 
R12  = 100  (1)  (0.75)  (1)  = 75 
R21  = 500  (1)  (0.975)  (1)  = 487.5 
R22  * 500  (1)  (0.75)  (1)  = 375. 

The  cost  associated  with  MTBF  p is  Mi  = $2500.  From  the  relationships  shown  in  Figure  13 
the  cost  associated  with  MTBFp2  may  Be  estimated  as  M2  = (1.5)  ($2500)  = $3750.  A judge- 
ment has  been  made  here  that  the  cost  of  TX  devices  is  approximately  one  and  one-half 
times  the  cost  of  JAN  devices  for  the  mix  of  parts  in  this  equipment. 

For  both  R^  and  Rn2>  Che  LET  setup  time  is  4 hours  at.  $8,  and  cycle  labor  is  4 hours 
each  at  $7.  Therefore, 

LLET1  = 528  (6)  + $32  = 8200 
LLeT-»  = $28  (2)  + $32  = $88. 

For  , the  following  conditions  are  given: 

Incoming  inspection  is  30  percent  sample  testing  and  the  corresponding  material 
burden  rate  is  11  percent. 

Lassv  is  60  hours  at  a direct  labor  rate  of  $6. 

Ltest  *s  ^0  hours  at  a direct  labor  rate  of  $8. 

L . is  2 hours  at  a direct  labor  rate  of  $10. 
sup 

= Lassya  + Ltesta  + Lsupa  f LLET1 

= $300  + $400  + $20  + $200  = $980 

Therefore , 

UPCU  = (Mt  (1  + 0.11)  + Lx  (1  + 0L))  (1  + G&A)  (1  + feel  . 

Arbitrary  values  are  assumed  for  0i , C&A,  and  fee  since  thev  are  constant  for  all  trade- 
offs. 

0L  = 1.5 

G&A=  0.2 


fee=  0.1. 


Then 


UPCU  = ($2500(1.11)  + $980  (2.5))  (1.2)  (1.1)  = $6897. 

Substituting  the  cost  of  material  associated  with  MTBFp2, 

UPC21  = (M2  (1  + 0.11)  + Lx  (1  + 0L))  (1  + G&A)  (1  + fee) 

= ($3750  (1.11)  + $980  (2.5))  ( 1 . 2) (1 . 1)  = $8728. 

For  R^,  tw0  situations  are  considered,  and  hence,  two  cost  results  for  the  same  reli- 
ability result.  The  first  situation  is  identical  to  except  for  the  specified  LET. 

Ly  ^assya  + ^testa  + ^supa  + ^LET2 


$360  + $400  + $20  + $88  = $868 


Therefore 

UPC1n 


UPC12a  = (M1  (1  + °-U)  + Ly  d + 1-5))  d + °-2>  d + 0-1) 

= ($2500  (1.11)  + $868  (2.5))  (1.2)  (1.1)  = $6527 

Upc22a  = (»2  d.ll)  + Ly  (2.5))  (1.2)  (1.1) 

= ($3750  (1.11)  + $868  (2.5))  (1.2)  (1.1)  = $8359. 

The  second  situation  for  Rm2  is  with  incoming  inspection  changed  to  100  percent  functional 
testing  and  a corresponding  material  buPden  rate  of  13  percent.  As  a result,  the  break- 
down of  labor  is 

Lassyb  = 40  hours  at  56 
Lfestb  = 20  hours  aC  $8 
LSUpb  = hours  at  $10. 

The  reduction  in  assembly  and  test  labor  is  partially  due  to  the  more  thorough  incoming 
testing  and  partially  due  to  increased  quality  support  such  as  vendor  surveillance  and 
failure  analysis. 

The  new  labor  total  is  given  by 

I'z  ^assyb  + ^testb  + ^supb  + H.ET2 

= $240  + $160  + $100  + $88  = $588 

and  correspondingly 

Upci2b  = (M1  d- 13)  + Lz  (2.5))  (1.2)  (1.1) 

= ($2500  (1.13)  + $588  (2.5))  (1.2)  (1.1)  = $5669 
Upc22b  = <M2  (1.13)  + L?  (2.5))  (1.2)  (1.1) 

= ($3750  (1.13)  + $588  (2.5))  (1.2)  (1.1)  = $7534. 

RANKED  ORDER  SUMMARY 

R2i  = 487.5  hours  UPC21  = $8728 

Rtt.-  375.0  hours  UPC00  = $8359 


21  = 487.5 

hours 

UPC21  = $8728 

22a=  375.0 

hours 

UPC22a=  $8359 

22b=  375.0 

hours 

UPC22b=  57534 

H=  97.5 

hours 

UPCU  = $6897 

12a=  75'° 

hours 

Upci2a=  $6527 

12b-  75. 0 

hours 

UPC12b=  55569 

The  results  shown  here  are  of  no  particular  significance,  because  the  values  selected 
for  the  variables  may  or  may  not  be  representative.  The  intention  of  this  example  is  to 
illustrate  the  application  of  the  relationships  developed  to  the  reliability  versus  unit 
production  cost  trade-off  analysis. 

TRADE-OFF  MODEL  TEST:  As  a test  of  the  validity  of  the  reliability  model,  the  data  from 
~"a  multiplex  set  I"s  used  as  an  example  exercise.  The  data  given  for  this  equipment  in- 
cluded the  reliability  mean  time  between  failure  prediction,  some  details  of  the  reli- 
ability growth  development  program,  and  the  results  and  conditions  of  production  reli- 
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ability  testing.  As  in  the  earlier  example,  the  post  deployment  or  logistics  influence 
is  normalized  by  setting  the  total  Rg  + Rj  + Rd  equal  to  unity.  Recalling  the  reli- 
ability equation 

R = MTBFp  (Rg)  (RJ  (Rg  + Rf  + Rd) . 

first,  substitute  the  predicted  mean  time  between  failure  of  the  multiplex  set 
MTBFp  = 4945  hours. 

Second,  estimate  Rg.  Since  a very  comprehensive  growth  program  of  greater  than  10,000 
hours  has  been  conducted,  it  may  be  assumed 


Third,  estimate  R^.  When  a burn-in  screen  without  temperature  cycling  is  used,  the  stress 
that  induces  failure  where  manufacturing  defects  exist  is  much  less,  and  faults  will  be 
detected  at  a lesser  rate.  One  cycle  of  a typical  temperature  cycling  screen  will  take 
six  to  eight  hours  to  complete.  Therefore,  in  terms  of  exposure  time  only,  48  hours  of 
burn-in  equates  to  approximately  six  to  eight  cycles.  Choosing  Ec  = 0.1  and  n = 6, 
solve  for  R^ 

Rm  = 1 - ( 1 - Ec)n  = 1 - (1  - 0.1)6  = 0.47 
The  resulting  predicted  reliability  is  then 

Kl  = 4945  (1)  (0.47)  (1)  = 2324  hours. 

This  compares  favorably  with  the  observed  failure  rate,  calculated  at  the  60  percent 
confidence  level,  of  2716  hours. 

A trade-off  was  considered  where  less  expensive,  lower  quality  parts  would  be  sub- 
stituted and  more  rigorous  screening  conducted  on  the  completed  assemblies.  For  this 
second  configuration,  all  integrated  circuits  would  be  specified  to  requirements  of 
MIL-M-38510,  class  C,  and  discrete  semiconductors  to  JAN.  The  resulting  reliability 
prediction  in  accordance  with  MIL-HDBK- 217B  was  2060  hours.  The  new  screening  require- 
ment for  completed  assemblies  included  six  cycles  of  temperature  cycling  from  C to  50 
degrees  centigrade  combined  with  2.2g  vibration  at  60  Hertz.  It  was  assumed  that  these 
variations  would  have  no  effect  upon  the  results  achieved  by  reliability  development 
growth  testing.  The  alternate  reliability  was 

R2  = 2060  (1)  (0.86)  (1)  = 1770  hours. 

The  actual  cost  of  the  multiplex  equipment  gives  a UPC^  corresponding  to  R^ . Recent 
average  production  values  are 


Material,  M^ 

= 

$7400 

Labor, 

= 

$3160 

°m 

= 

0 (redistributed 

into 

labor 

burden) 

°1 

= 

150  percent 

G&A 

= 

20  percent 

Fee 

= 

10  percent 

UPCr 

- 

(ML  (1  + om)  + ll 

(1 

+ op) 

(1  + G&A)  (1  + fee) 

= 

($7400  + $3160  (2 

.3)) 

(1.2) 

(1.1)  = $20,196. 

The  alternate  cost,  UPC2  corresponding  with  R2  is 
M2  = $5300 

L„  = $3760  (increased  because  of  added  setup  time  for  screening  and  more 

test  and  repair  due  to  lower  quality  parts,  i.e.,  more 
defects  detected) 

UPC2  = ($5300  + $3760  (2.5))  (1.2)  (1.1)  - $19,404. 

SUMMARY  DATA 


: 


Ml 


N 


■ 


Alternate  1 
Alternate  2 


Reliability 
2324  hours 
1770  hours 


Cost 

$20,196 

$19,404 


Alternate  2 was  not  selected  because  although  it  results  in  a four  percent  cost  re- 
duction, the  reliability  is  reduced  by  almost  24  percent  to  a value  that  did  not  meet 
the  requirement. 
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This  example  demonstrates  the  utility  of  the  estimating  relationships  as  well  as 
showing  excellent  correlation  between  predictions  and  actual  results  observed. 

OPERATION  AND  SUPPORT  COST  MODELS:  An  essential  ingredient  in  an  LCC  model  is  the  de- 
velopment  of  operation  and  support  (O&S)  cost  models.  These  models  are  basically  an- 
alytical in  nature.  That  is,  an  operations,  maintenance  and  support  scenario  is  first 
developed,  and  around  this  scenario,  a complete  cost  model  is  developed.  Examples  of 
relationships  explored  in  the  development  of  such  models  are:  the  quantitative  effect 
of  reliability  on  the  number  of  maintenance  actions  and  spare  parts  requirements;  the 
effect  of  maintainability  on  the  number  of  maintenance  man-hours  required,  and  the  man- 
power required  per  maintenance  action;  and  the  development  of  materiel  costs,  and  costs 
per  man-hour,  for  the  required  maintenance  activities.  A virtual  plethora  of  such 
models  have  been  developed  in  the  past;  one  report  published  almost  eight  years  ago 
contained  46  different  models  ("Using  Logistics  Models  in  System  Design  and  Early  Plan- 
ning", the  Rand  Corporation,  Santa  Monica,  California,  February  1971). 

Operating  and  maintenance  costs  are  the  sum  of  initial  and  recurring  costs  over  the 
system  life  cycle.  The  O&M  cost  model  is  therefore  the  summation  of  every  significant 
cost  element.  For  illustrative  purposes,  the  following  cost  elements  are  extracted  from 
the  O&S  modeling  portion  of  a United  States  Department  of  Defense  Publication,  "Life 
Cycle  Costing  Guide  for  System  Acquisitions”,  (Interim)  January  1973.  The  guide  lists 
the  following  as  pertinent  O&M  cost  factors: 


1.  Operational  Personnel  and  Consumables  Costs 

a.  Personnel 

b.  Consumables 

2.  Training  Costs 

a.  Initial  and  Replacement  Training 

b.  Recurring  Training 

3.  Maintenance  Costs 

a.  Organizational 

b.  Intermediate 

c.  Depot  (system  level) 

d.  Depot  (subsystem  or  component  level) 

4.  Facilities 


5. 

6. 

7. 

8. 
9. 

10. 

11. 

12. 


Initial  Government  Materiel  and  Services 

Support  and  Test  Equipment 

Data 

Initial  Spares  and  Repair  Parts 
Salvage  and  Disposal 

Initial  and  Replacement  Transportation 
Supply  Management 
Development  and  Test 


Equations  provided  for  the  first  four  items  are: 

1.  Operational  Personnel  and  Consumables  Costs 
a.  Personnel 

Y NT  NS 

COP  = 1 Dv  1 I (PR  ) (CP  .) 
k=l  j=l  s=l  sJk  SJ 

where : 

COP  = Life  cycle  operational  personnel  cost. 

CP  s = Average  annual  cost  (including  all  pay,  allowances,  medical  care, 
dental,  retirement,  etc.)  of  a man  of  skill  "s"  and  type  "j". 

D^  = Discount  factor  for  year  "k"  to  generate  a "present  value". 

NS  = Number  of  combinations  of  different  skills  and  levels  within  skill 
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NT  = Number  of  types  of  personnel 

PR  = Number  of  required  personnel  (including  the  Government  provided  fac- 
J tor  to  account  for  leave,  sickness,  etc.)  of  skill  " s"  and  type  "j" 
in  year  "k" . (Wherever  this  symbol  occurs,  it  refer  to  personnel 
relevant  to  that  equation) . 

Y = System  operating  life  cycle  (to  the  nearest  year) . 
j = Type  of  personnel  (civilian/military/etc.) 
k = Year  in  life  cycle  of  the  system, 
s = Skill  type  and  level. 


Double  counting  of  costs  where  operating  personnel  perform  maintenance  must  be  avoided, 
b.  Consumables 

Y NC 

COC  = 1 D,  I (RC.)  (CUC.)  (HC.,) 
k=l  i=l  1 1 1K 

where : 

COC  = Life  cycle  operational  consumables  cost. 

CUCp  = Cost  of  operational  consumable  item  "i"  per  unit  consumed,  including 
cost  of  transportation  to  point  of  use, 

HCj^  = Programmed  operational  use  time  (hours  of  utilization) , in  year  "k" 
of  the  ships,  aircraft,  etc.,  which  consume  item  "i". 

NC  = Number  of  consumable  items. 

RC^  = Consumption  rate  (units/hour  of  utilization)  of  consumable  item  "i". 
(Hours  of  utilization  must  be  compatible  with  those  in  HC^.) 

i = Item  number  ("i"  will  be  used  throughout  this  Guide  to  identify 
consumable  and  recoverable  items) . 

Other  symbols  are  as  previously  defined. 

Items  visualized  under  the  operating  consumables  category  are  *POL, 
electrical  power,  hydraulic/pneumatic  power,  heating,  and  cooling  energy,  nuclear  power, 
and  consumable  materials.  Although  the  equation  is  generally  applicable  to  each  of  these 
categories,  it  may  require  somewhat  different  inclusions  within  the  specific  terms.  For 
example,  POL  consumption  will  consider  cost  per  unit  of  consumption  (pounds,  gallons, 
etc.)  times  utilization  hours.  In  the  case  of  nuclear  power,  capital  outlays  may  be  re- 
quired at  specific  dates  (e.g.,  core  replacement),  and  these  will  be  discounted  from 
their  respective  dates. 


2.  Training  Costs 

a.  Initial  and  Repacement  Training 


Y NT  NS 

CIT  = Z Dk  Z Z ((Cl.,)  (PR 

k=l  j = l s = l ’ SJk 


PA 


sjk 


PFsjk>  + 


(CUsj>  (PAsjU» 


where : 


CIT  = Total  initial  and  replacement  training  cost  of  personnel. 


CIqi  = Induction  and  initial  training  cost  per  man  (entire  cost,  indue 
J pay  and  allowances,  to  bring  a man  into  the  service  and  up  to  the 
required  skill  type  and  level  "s"  for  personnel  type  "j") . 


CU  . = Update  training  cost  per  man  to  bring  available  personnel  up 
s 1 required  level  for  skill  type  and  level  "s"  and  personnel  ' 

PA  Number  of  available  personnel  of  skill  tvpe  and  level  "■ 

h sonnel  type  "j"  that  do  not  require  initial  training  ii 

require  update  training. 

PF  ..  Number  of  personnel  of  skill  type  and  level  "s"  and  ■ 

"j"  that  are  available  and  fully  trained  in  vear  ”k' 


Other  symbols  are  as  previously  defined. 


* POL  is  defined  as  power,  oil,  lubricants,  etc. 
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Notes:  1.  Personnel  types  and  skill  levels  will  include  categories  cited  as  operational 
personnel  in  equation  la,  and  all  other  categories  used  in  this  system. 

2.  Also  note  carefully  the  definition  of  PAgjk  and  PFgjk. 

3.  Personnel  must  be  available  to  work  on  the  system  under  consideration  (e.g., 
not  merely  available  within  the  Service  but  assigned  to  another  system). 

4.  PAgik  and  PF  . variations  from  year  to  year  include  a reflection  of  turn- 
over rates.  SJ*C 

b.  Recurring  Training 


y 

CRT  = 1 

k=l 


j=l  s=l 


<CR.jk>  ‘"sjk* 


where: 


CRT  = Life  cycle  recurring  training  cost. 

CR  ..  = Recurring  training  cost  in  year  "k"  to  maintain  the  proficiency  of  those 
J personnel  working  on  the  system,  for  skill  type  and  level  "s"  and  per- 
sonnel type  "j". 

Other  symbols  are  as  previously  defined. 

Note:  The  use  of  PF^^.  in  the  equation  for  recurring  training  is  based  on  the  premise 
that  those  personnel  entering  the  system  during  year  "k"  would  be  fully  trained 
and  would  not  require  recurrent  training  until  the  succeeding  year. 

3.  Maintenance  Costs:  In  the  maintenance  equations  below,  exclude  labor  costs  for 
those  tasks  which  are  accomplished  by  operating  personnel  costed  above.  (Operational 
Personnel  Costs). 


Organizational 


CMO  = E ((D.  ) (HO)  (CLO)  + CCO  + CRO) ) 
k=l  K 


(HUP  ) 
k 


where: 


where : 


CMO  = Life  cycle  material  and  labor  cost  for  organizational  maintenance. 

CCO  = Total  cost  of  consumable  material  used  in  organizational  level  main- 
tenance during  the  operational  test  period.  (Includes  the  cost  of 
items  identified  as  "discard-at-failure") . 

CLO  = Average  organizational  level  maintenance  labor  cost  per  man-hour  to 
repair  items  which  were  removed  during  the  operational  test  period 
(direct  and  indirect) . 

CRO  = Total  cost  of  recoverable  material  condemned  at  the  organizational 
level  during  the  operational  test  period. 

HO  = Total  maintenance  labor  man-hours  used  in  performing  organizational 
level  maintenance  during  the  operational  test  period. 

HT  = Total  utilization  hours  of  ships,  aircraft,  tanks,  etc.,  during  the 
operational  test  period. 

HUPk  = Total  operational  hours  of  utilization  programmed  for  this  entire 
force  of  aircraft,  ships,  tanks,  etc.,  in  year  "k" . 

= 12  ((RUO)  (NOU)  + (RUT)  (NTU))  k 


NOU  = Number  of  individual  operational  aircraft,  ships,  tanks,  etc. 

NTU  = Number  of  individual  training  and  other  nonoperational  aircraft, 
ships,  tanks,  etc. 

RUO  = Operational  utilization  rate  (hours/month)  per  aircraft,  ship,  tank 
etc. 

RUT  = Training  and  other  nonoperational  utilization  rate  (hours/Month) 
per  aircraft,  ship,  tank,  etc. 


Other  symbols  are  as  previously  defined. 
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b.  Intermediate 
Y 

CM  I = 1 D. 
k-1  K 


Repair  Costs 
((HI)  (CLI)  + CCI)) 


HUP 


HT 


_+  I 
i-1 


Replenishment  Costs* 
NR  HUP. 

k + 


CRIt  ( 


HT~ 


NR 

I 

1=1 


Pipeline  Costs* 

(CPI.)  (HSI.  , - HSI  ) 

± L_k^l 

HT 


+ 


NT 

Z 

i = l 


Transportation  Costs 
(NTIt)  ( CT 1 1 ) (JljJpL) 


*The  sum  of  pipeline  costs  and  replenishment  costs  (for  intermediate  and  depot  levels 
combined)  for  each  item  "i"  in  any  year  cannot  be  less  than  zero.  If  negative,  replace 
all  terms  involved  by  zero  for  that  item. 

where : 


CMI  = Life  cycle  material,  labor  and  transportation  cost  for  intermediate 
maintenance. 

CCI  - Total  cost  of  consumable  material  used  in  intermediate  level  main- 
tenance to  repair  units  which  were  removed  during  the  operational  test 
period. 

CLI  = Average  intermediate  level  maintenance  labor  cost  per  man-hour  to  re- 
pair items  which  were  removed  during  the  operational  test  period 
(direct  and  indirect). 

CPI|  = Unit  acquisition  cost  of  recoverable  item  "i"  multiplied  by  the  number 
of  times  that  item  was  removed  during  the  operational  test  period  and 
ultimately  repaired  at  the  intermediate  level. 

CRIj_  = Unit  acquisition  cost  of  recoverable  item  "i"  multiplied  by  the  number 
of  times  that  item  was  removed  during  the  operational  test  period  and 
ultimately  condemned  at  the  intermediate  level. 

CTI^  = Average  round  trip  transportation  cost  (including  packaging,  administra- 
tion and  scheduling  from  removal  to  reinstallation)  per  unit  of  item 
"i"  removed  during  the  test  period  and  sent  to  intermediate  level, 
and  condemned  or  repaired  at  that  level. 

HI  = Total  maintenance  labor  man-hours  used  in  performing  intermediate 

level  repair  on  the  units  that  were  removed  during  the  operational  test 
period. 

HSI^  k = The  number  of  total  operating  hours  for  the  entire  force  of  aircraft, 

’ ships,  tanks,  etc.,  programmed  during  the  intermediate  repair  cycle 
time  for  item  "i"  in  the  year  "k"  (where  cycle  time  covers  the  period 
of  time  from  removal  to  reinstallation  of  item  "i",  based  on  first-in, 
first-out  processing). 

NR  - Number  of  different  recoverable  items  in  the  system. 

NTI^  - The  number  of  units  of  item  "i"  that  were  removed  during  the  operational 
test  period  and  sent  to  the  intermediate  level,  and  repaired  or  con- 
demned at  that  level. 

Other  symbols  are  as  previously  defined, 
c.  Depot  (System  Level) 

COD  = ] 12  (D.)  jHiU  NTU_  (COH) 

k-1  k M0D  KTDk 

where: 

COD  = Life  cycle  cost  of  system  overhaul  at  depot. 

COH  « Total  cost  (labor,  overhead,  round-trip  transportation,  and  material)  of 
individual  aircraft,  ships,  tanks,  etc.,  of  each  system  overhaul. 

MOD  = Calendar  months  between  overhauls  of  operational  ships,  aircraft, 
tanks,  etc. 


MTD  - Calendar  months  between  overhauls  of  training  and  other  nonoperat ional 
ships,  aircraft,  tanks,  etc. 


Other  symbols  are  as  previously  defined, 
d.  Depot  (Subsystem  or  Component  Level) 

Y Repair  Costs  Hup 


Replenishment  Costs* 


CMD  - T.  Dk  ((HD)  (CLD))+  CCD  — ^ + Z CRDX 

k=l  i=l 


Pipeline  Costs* 


Transportation  Costs* 


7 (_CPDj)  (HSD_Uu=JSPt>  k-1)  + f (HTD  , (CTDi)(^) 

i=l  HT  i=l  HT 

*The  sum  of  pipeline  costs  and  replenishment  costs  (for  intermediate  and  depot  levels 
combined)  for  each  item  "i"  in  any  year  cannot  be  less  than  zero.  If  negative,  replace 
all  terms  involved  by  zero  for  that  item. 

where : 


CCD  = Total  cost  of  consumable  material  used  in  deoot  level  maintenance  to 
repair  units  which  were  removed  during  the  operational  test  period. 

CLD  = Average  depot  level  maintenance  labor  cost  per  man-hour  to  repair 

items  which  were  removed  during  the  operational  test  period  (direct 
and  indirect) . 

CMD  = Life  cycle  material,  labor  and  transportation  cost  for  depot  mainten- 
ance at  subsystem  or  component  level. 

CPD^  = Unit  acquisition  cost  of  recoverable  item  "i"  multiplied  by  the  number 
of  times  that  item  was  removed  during  the  operational  test  period  and 
ultimately  repaired  at  the  depot  (subsystem  or  component  level). 

CRD ^ = Unit  acquisition  cost  of  recoverable  item  "i"  multiplied  by  the  number 
of  times  that  item  was  removed  during  the  operational  test  period  and 
ultimately  condemned  at  the  depot  (subsystem  or  component  level) . 

CTD^  « Average  round  trip  transportation  cost  (including  packaging,  adminis- 
tration and  scheduling,  from  removal  to  reinstallation)  per  unit  of 
item  "i"  removed  during  the  operational  test  period  and  sent  to  depot 
leve 1 . 

HD  = Total  maintenance  labor  man-hours  used  in  performing  depot  level 
repair  on  the  units  that  were  removed  during  the  operational  test 
period . 

HSD.,  = The  number  of  total  system  operating  hours  programmed  during  the 
1 depot  repair  cycle  time  for  item  "i"  in  year  "k”  (where  cycle  time 
covers  the  periods  of  time  from  removal  to  reinstallation  of  item 
"i",  based  on  first-in,  first-out  processing). 

NTD|  = The  number  of  units  of  item  "1"  that  were  removed  during  the  operational 
test  period  and  sent  to  the  depot. 

Other  symbols  are  as  previously  defined. 

THE  SAVE  PROGRAM:  A more  recent  development  reported  in  the  proceedings  of  the  IEEE  1978 
National  Aerospace  and  Electronic  Conference  NAECON  78,  Vol  3,  is  "System  Avionics  Value 
Estimation  (SAVE)";  a new  tool  for  Logistics  and  Support  cost  analysis.  This  is  a col- 
lection of  five  special  purpose  computerized  logistics  and  support  cost  models  integrated 
within  an  interactive  network.  Developed  by  the  Air  Force  Avionics  Laboratory,  it  at- 
tempts to  avoid  problems  of  logical  incompatibility,  inconsistent  input/output  and  a lack  ol 
overall  guidance  which  arise  when  several  different  models  are  used  in  one  procurement. 

It  is  mentioned  here  as  an  example  of  the  reduction  of  the  myriad  of  O&M  cost  models 
to  one  computerized  analytical  aid.  Its  approach  is  worth  emulating. 

RELIABILITY  AS  A CAPITAL  INVESTMENT:  Life  cycle  cost  modeling  permits  one  to  consider 
costed  activities  as  Investments,  comparing  the  acquisition  costs  of  any  activity  (invest- 
ment) to  potential  savings  in  O&M  costs  (returns).  One  example  of  this  is  "Reliability  as 
a Capital  Investment"  published  in  the  proceedings  of  the  1974  Annual  Reliability  and 
Maintainability  Symposium,  Los  Angeles,  California.  The  following  is  the  description 
given  in  the  paper  for  computing  the  return  on  an  investment  in  reliability  activity 
during  acquisition. 


V 
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Return  of  investment  (ROI)  is  simply  the  annual  return  divided  by  the  investment 
cost.  In  the  context  of  this  paper: 

ROI  1 = C0~CN  (1) 

CR 


where : 

CO  - annual  maintenance  costs  of  a system  procured  without  a reliability  program. 
CN  = annual  maintenance  cost  of  a system  procured  with  a reliability  program. 

CR  = cost  of  the  reliability  program. 

Further  expanding  the  terms: 

CO  = V(l)  V(2)  V(3)  V(4)  -12  -V(5)  (2) 

where : 

V(l)  » field  failure  rate  of  a system  procured  without  a reliability  program. 

V(2)  = number  of  parts  in  the  system 
V(3)  = number  of  systems  installed. 

V(4)  = operating  hours  per  month. 

V(5)  = cost  to  repair  a failure. 

CN  - V(6)  V(2)  V(3)  V(4)  -12  ‘V(5)  (3) 

where : 

V(6)  = field  failure  rate  of  a system  procured  with  a reliability  program. 
Remaining  terms  as  defined  above. 

CR  = (CM  + CL  + CT)  (1  + V(26) ) (1  + V(27)  (4) 


where : 


CM  - cost  of  materials  (i.e.,  parts  screening) 

CL  - cost  of  labor  for  reliability  activities  excluding  test  related  activity. 

CT  = costs  of  all  reliability  tests  including  labor  and  failure  analysis  activity. 

V(26)  * G&A  overhead  factor. 

V(27)  = profit  factor. 

Further  expanding  the  terms  of  Equation  5: 

CM  = V(2)  V(3)  V(7)  (1  + V(8) ) V(33)  (5) 

where  (new  terms  only) : 

V(7)  = cost  of  screening  per  part. 

V(8)  ” materials  overhead  factor. 

V(33)  = ratio  of  semiconductors  to  total  parts. 

NOTE:  This  equation  considers  only  the  cost  of  semiconductor  screens  (i.e.,  micro- 

circuits  per  MIL-STD-883  and  other  semiconductors  per  TX  (specifications).  Other 
populous  high  reliability  parts  (e.g.,  parts  per  ER  specifications)  are  available 
without  extra  cost  over  standard  parts. 

CL  » V(9)  V(10)  V(ll)  V(2)  + V(12)  V(13)  V(14)  V(2)  (6) 

where : 

V(9)  * engineering  costs  per  hour. 

V(10)-  engineering  overhead  factor. 

V(ll)~  no.  reliability  engineering  hours  per  part. 

V(12)«  QC  inspection  costs  per  hour. 

V(13)=  factory  overhead  factor. 
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V(14)  - QC  inspection  man-hours/part  due  to  reliability  program. 

NOTE:  This  equation  computes  the  cost  of  routine  reliability  and  QC  inspection  efforts 
caused  by  the  reliability  program.  It  is  assumed  that  the  amount  of  effort  is 
proportioned  to  the  complexity  of  the  equipment. 

CT  = CABI  + CRET  + CRQT  + CSBI  (7) 

where: 

CABI  = costs  due  to  assembly  burn-in. 

CRET  - costs  due  to  reliability  evaluation  testing. 

CRQT  - costs  due  to  reliability  qualification  tests  (demonstration). 

CSBI  - costs  due  to  system  burn-in. 

Expanding  the  terms  of  Equation  7: 

CABI  = G-H  (8) 

when: 

G = total  hours  of  burn-in  required  for  special  assemblies  (e.g.,  power  supplies). 
H = cost  per  hour  for  test  technician  and  failure  analysis. 

Expanding  the  terms  of  Equation  8: 

G = V(2)  V(3)  V(15)  V(16)  (9) 

where : 

V(15)  = ratio  of  special  assemblies  to  total  parts. 

V(16)  = length  of  burn-in  per  assembly. 

H - V(32)  V(17)  V(13)  + V(6)  V(21)  V(18)  V(9)  V(10)  V(19)  (10) 

where : 

V(32)  = ratio  of  technical  time  to  burn-in  (assuming  that  continuous  monitoring 
is  not  required  for  assembly  burn-in). 

V(17)  - cost  of  test  technician  per  hour. 

V(21)  - ratio  of  failure  rate  experienced  in  burn-in  to  field  failure  rate 

(also  used  as  ratio  of  failure  rate  during  reliability  evaluation  test 
to  field  failure  rate) . 

V(18)  - number  of  parts  per  special  assembly. 

V(19)  “ engineering  hours  needed  for  failure  analysis  per  failure. 

Returning  now  to  Equation  7 : 

CRET  - V(20)  (AX)  (11) 

where : 

V(20)  = length  of  reliability  evaluation  test  (operating  hours) 

AX  “ cost  of  RET  per  hour. 

Expanding  AX: 

AX  - V(17)  V(13)  V(34)  + V(6)  V(21)  V(2)  V(9)  V(10)  V(19) 

where : 

V(34)  = ratio  of  technician  time  to  test  hours  assuming  continuous  monitoring 
and  a test  cycle  including  "equipment  off"  periods  which  do  not  count 
as  operating  time  (i.e.,  countable  test  time  is  less  than  calendar  time). 

All  other  terms  as  defined  above. 

Continuing  with  the  terms  of  Equation  7: 


CRQT  + (AA)  (AB) 


(12) 
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when: 


AA  « reliability  demonstration  test  time. 

AB  « cost  per  hour  for  reliability  demonstration  test. 

Expanding  terms  of  Equation  13: 

AA  “ 1 V (22)  V(23)  (13) 

V(<5)  V(2) 

where : 

V(22)  = the  ratio  of  specified  MTBF  (mean  time  between  failure)  to  field  MTBF. 
It  is  assumed  the  test  specified  MTBF  will  be  higher  than  the  field 
MTBF.  No  retest  is  assumed  as  the  reliability  evaluation  test  should 
have  provided  enough  data  to  allow  the  specified  MTBF  to  be  met  during 
the  test. 

V(23)  = ratio  of  required  test  hours  to  specified  MTBF. 

NOTE : The  remaining  terms  have  been  defined  and  provide  the  field  MTBF. 

AB  = V(17)  V(13)  V(34)  + V(6)  V(9)  V(10)  V(19)  V(2)  (13a) 

Where  all  terms  have  been  previously  defined. 

Concluding  Equation  7: 

CSBI  =(V(3)  V (24))(V(17)  V(13)  + V(9)  V(10)  V(19)  V(6)  V(25)  V(2))  (14) 


where : 


V(24)  = hours  of  burn-in  required  per  system. 

V(25)  = ratio  of  system  burn-in  failure  rate  to  field  failure  rate. 

All  other  terms  previously  defined. 

Rate  of  return  compares  investment  costs  to  annual  recurring  cost  savings.  Initial 
0&M  cost  savings  are  handled  by  subtracting  these  from  the  investment  cost.  For  example, 
the  paper  shows  that  a reduction  in  the  cost  of  initial  spares  procurement  can  be  con- 
sidered an  immediate  return  and  hence,  subtracted  from  the  cost  of  the  reliability  pro- 
gram. 

LCC  CONFIDENCE  INTERVALS : Given  the  appropriate  development,  production,  and  operation 
and  support  costs  for  a system,  one  can  then  sum  them  to  estimate  total  life  cycle  costs. 
It  is  obvious  that,  since  each  estimate  is  subject  to  some  degree  of  inaccuracy  or  error, 
the  total  LCC  will  be  sensitive,  in  different  degrees,  to  some  combination  of  the  inherent 
errors.  The  following  are  extracts  from  an  RADC  study  aimed  at  quantifying  model  sensi- 
tivity and  developing  procedures  for  estimating  confidence  intervals  for  LCC  models.  The 
work  is  still  in  progress  and  so  is  not  available  in  the  literature. 

APPROACHES  TO  HANDLING  UNCERTAINTY : The  benefit  derived  from  achieving  cost  visibility 
is  greatly  diminished  if  projections  are  frequently  wrong  and  the  costs  portrayed  are  not 
representative  of  reality.  As  a result,  methods  must  be  used  to  account  for  the  uncer- 
tainty involved  and  in  this  manner  develop  more  credible  projections  of  LCC. 

Complete  certainty  can  be  obtained  only  from  a historical  perspective  and  this  is  too 
late  to  assist  in  the  planning  of  the  project  at  hand.  However,  various  approaches  to 
handling  uncertainty  are  available  and  have  been  used  at  one  time  or  another.  Four  of 
these  approaches  are  briefly  discussed  below. 

Safety  Margins:  Perhaps  one  of  the  oldest  techniques  for  protecting  oneself  against 
uncertainty  is  to  provide  a margin  of  safety  in  the  estimates  of  critical  parameters. 
Quite  often,  the  worst  possible  value  of  a parameter  is  used  to  represent  the  projected 
outcome.  While  this  procedure  can  be  used  to  establish  an  upper  bound  on  possible  costs, 
it  can  work  to  the  disadvantage  of  a program  by  eliminating  certain  courses  of  action  as 
too  expensive  when,  in  fact,  they  may  be  too  expensive  only  if  the  worst  possible  set  of 
circumstances  occur. 

Expected  Values:  Basing  decisions  on  an  expected  outcome  may  well  be  one  of  the 
most  common  decision-making  procedures  in  use  today.  The  advantages  of  this  approach  is 
that  it  requires  only  a single  estimate  representing  what  the  analyst  feels  will  actually 
happen.  Also,  if  the  decision  is  made  many  times,  the  expected  return  will  be  realized. 
The  disadvantage  is  that  the  degree  of  uncertainty  that  may  affect  the  final  decision  is 
not  reflected  and  it  may  not  be  possible  to  make  the  decision  many  times. 

Sensitivity  Analysis:  Sensitivity  analysis  is  another  fairly  common  way  of  treating 
uncertainty.  Use  of  this  approach  calls  for  one  value,  such  as  reliability,  to  be  varied 
over  its  range  while  other  values  are  held  constant.  This  makes  it  possible  to  determine 
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Che  degree  of  sensitivity  that  a LCC  estimate  has  to  various  input  parameters.  The 
technique  works  best  when  uncertainty  exists  about  only  a few  parameters  and  when  the 
parameters  are  independent  (i.e.,  the  effect  on  costs  of  the  various  parameters  is 
separable).  However,  when  uncertainty  exists  in  a large  number  of  input  variables,  the 
number  of  possible  combinations  of  values  becomes  very  large  and  may  prohibit  one  from 
analyzing  all  possibilities.  Also,  the  likelihood  of  some  combinations  occurring  will 
be  much  less  than  the  likelihood  of  other  combinations.  Straightforward  sensitivity 
analysis  will  not  reflect  these  changing  probabilities  and  a LCC  resulting  from  extreme 
values  of  all  input  parameters  will  appear  just  as  likely  as  one  resulting  from  the  use 
of  average  input  values. 

Monte  Carlo  Simulation:  Monte  Carlo  simulation  provides  yet  another  way  of  accounting 
for  the  uncertainties  in  input  parameters  of  a LCC  analysis.  Such  an  approach  requires 
that  information  concerning  input  variables  be  available  to  the  LCC  analysis  model  in  the 
form  of  probability  distributions.  This  means  that  either  the  raw  input  data  must  con- 
sist of  probability  distributions  or  else  the  model  itself  must  convert  deterministic 
input  data  into  a probability  distribution. 

Likewise,  the  model  output  consists  of  a distribution  of  life  cycle  costs. 

From  this  output  distribition  it  is  possible  to  determine  both  the  expected  life  cycle 
cost  and  the  uncertainty  associated  with  the  estimate. 

There  are  several  advantages  of  a Monte  Carlo  simulation  approach  to  LCC  analysis 
over  the  more  traditional  sensitivity  analysis  approach.  First,  and  perhaps  most  im- 
portantly, it  provides  information  concerning  the  probability  of  occurrence  of  specific 
life  cycle  costs.  LCC  values  arising  from  a sensitivity  analysis  are  simply  point  estim- 
ates of  a life  cycle  cost  and  nothing  is  formally  given  about  the  likelihood  of  any 
particular  value  occurring.  Common  sense  tells  us  that  the  occurrence  of  extreme  values 
in  two  or  more  parameters  simultaneously  is  not  as  likely  as  the  simultaneous  occurrence 
of  more  moderate  values.  However,  this  difference  in  probability  is  not  quantified  dur- 
ing a normal  sensitivity  analysis  where  it  can  be  when  the  analysis  is  conducted  by 
Monte  Carlo  simulation. 

Second,  it  relieves  the  analyst  of  the  responsibility  of  deciding  what  combina- 
tions of  parameter  values  to  use  in  calculating  a life  cycle  cost.  Deciding  which  com- 
binations of  values  warrant  investigation  and  then  making  the  proper  calculations  can 
be  a burdensome  task  when  there  are  a large  number  of  input  parameters. 

Because  of  these  advantages,  a Monte  Carlo  approach  to  LCC  analysis  will  be 
discussed  further. 

A Simulation  Approach  to  Life  Cycle  Cost  Analyses:  There  are  several  major  consider- 
ations  that  should  shape  the  development  of  any  simulation  model  being  designed  for  use 
in  conducting  a LCC  analysis.  First,  it  must  be  as  accurate  as  possible.  Next,  it 
should  be  simple  to  operate  and  last,  the  output  should  be  structured  so  that  it  is 
easy  to  understand.  It  is  only  by  following  these  guidelines  during  model  development 
that  proper  visibility  can  be  given  to  the  costs  incurred  by  an  equipment  item  over  its 
life. 

Model  Accuracy:  Model  accuracy  is  largely  a function  of  three  things.  These  are 
. Validity  of  the  input  data. 

. Completeness  of  the  cost  categories. 

. Treatment  of  the  costs  within  the  model. 

Validity  of  the  input  data  is  in  many  respects  an  intangible  that  is  difficult  to 
quantify.  It  is  dependent  both  on  the  ability  of  the  analyst  to  extract  and  formulate 
the  data  as  well  as  on  the  availability  of  data  sources.  As  a result,  the  validity  of 
input  data  can  vary  greatly  from  application  to  application.  To  accommodate  this 
situation,  data  input  requirements  for  a model  should  be  as  limited  as  possible  while 
still  enabling  the  model  to  meet  its  output  objectives.  This  point  is  pursued  further 
in  the  discussion  on  model  completeness  immediately  below. 

Insuring  that  the  cost  categories  of  a model  are  adequately  complete  requires  a 
great  deal  of  attention.  The  model  must  contain  a sufficient  number  of  categories  to  be 
complete  and  also  provide  sufficient  detail  for  control  and  decision-making  purposes  when 
evaluating  design  trade-offs.  At  the  same  time,  the  number  of  categories  must  be  re- 
stricted in  order  to  retain  simplicity.  The  exact  structure  of  the  cost  categories 
should  represent  a balance  between  accuracy  and  simplicity.  The  ultimate  model  structure 
is  therefore  somewhat  dependent  on  the  use  to  be  made  of  the  model  results. 

The  use  of  CERs  introduces  a unique  problem  when  insuring  that  the  cost  categories  of 
a model  afford  complete  coverage.  CERs  are  not  always  straightforward  and  the  original 
cost  data  upon  which  the  relationships  are  based  may  not  be  readily  available  for  analy- 
sis and  verification.  This  means  that  the  use  of  CERs  should  be  restricted  to  those  that 
are  thoroughly  documented  or  that  can  be  verified  in  some  manner. 

Treatment  of  costs  within  the  model  may  consist  of  simply  summing  up  costs  in  dif- 
ferent categories.  The  main  caution  to  be  observed  on  this  point  during  model  develop- 
ment is  to  make  sure  that  all  relevant  costs  are  included  and  that  none  are  included 
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more  than  once.  This  is  particularly  true  if  CERs  are  used  to  estimate  cost.  As  stated 
earlier,  the  model  builder  must  know  in  detail  what  costs  are  covered  by  a specific  CER. 
One  should  also  be  willing  to  judge  whether  or  not  the  history  represented  by  the  old 
data  points  used  to  structure  the  CER  will  repeat  itself  in  the  application  being  analyzed. 

One  other  topic  that  needs  to  be  considered  when  discussing  the  treatment  of  costs 
within  the  model  is  the  Monte  Carlo  sampling  procedure.  A Monte  Carlo  Procedure  operates 
by  taking  samples  of  possible  values  of  input  variables  and  combining  them  as  dictated 
by  the  logic  internal  to  the  model  to  produce  a final  result.  The  input  variables  on 
most  models  are  independent  which  means  that  any  value  of  one  variable  can  be  selected 
and  combined  with  any  value  of  another  variable.  This  is  not  always  the  case  when  deal- 
ing with  a cost  model  and  requires  special  attention  to  insure  that  the  model  yields 
accurate  results. 

The  problem  of  dealing  with  dependent  input  variables  (i.e.,  dependent  cost  categories) 
can  be  solved  in  two  ways.  The  first  is  to  combine  all  cost  into  more  highly  aggregated 
groupings  which  are  no  longer  dependent.  This  is  not  desirable  if  the  model  is  to  be 
used  to  evaluate  design  trade-offs  because  the  necessary  cost  detail  will  be  lost. 

The  second  method  involves  modifying  the  normal  sampling  procedure  used  by  a Monte 
Carlo  model.  The  cost  categories  must  first  be  ranked  in  a priority  or  precedence  order. 
This  order  can  change  from  sample  to  sample,  but  must  remain  constant  during  a single 
sampling  operation.  The  model  will  then  obtain  sample  values  of  all  input  variables  in 
the  order  dictated  by  the  priority  listing.  The  first  value  obtained  will  be  selected 
from  the  entire  range  permitted  of  that  variable.  However,  from  that  point  on  each 
subsequent  variable  will  be  checked  to  determine  if  a dependence  exists  between  it  and 
the  variable  previously  selected.  If  a dependency  does  exist,  the  potential  range  of 
the  second  variable  will  be  restricted  to  reflect  the  dependency  relationship.  The  type 
of  restriction  imposed  is  dynamic  and  will  vary  from  sample  to  sample  during  the  simula- 
tion run.  Model  complexity  will  be  greatly  reduced  if  the  priority  list  can  he  structured 
so  that  each  variable  will  have  to  be  checked  for  dependency  only  against  the  one  pre- 
ceding it  on  the  list. 

Simplicity  of  Operation:  Simplicity  of  model  operation  goes  a long  way  toward  es- 
tablishing its  eventual  usefulness.  A model  that  is  too  complex  to  operate  will  not  be 
used  enthusiastically  and  as  a result,  may  fall  into  disuse.  As  far  as  an  analyst  is 
concerned,  simplicity  of  operation  is  determined  by  the  requirements  imposed  on  structur- 
ing the  input  data.  Care  must  be  exercised  in  this  regard  when  dealing  with  Monte  Carlo 
simulation  models.  Traditional  thinking  says  that  better  results  will  be  obtained  when 
more  precisely  structured  input  data  are  used.  With  Monte  Carlo  models  this  often  means 
that  input  data  must  be  structured  in  the  form  of  a histogram  or  probability  distribution. 
This  is  a severe  requirement  and  one  that  can  only  be  carried  out  with  difficulty  for 
typical  life  cycle  cost  data. 

An  alternative  to  this  approach  is  to  structure  the  model  so  that  it  fits  a 
known  probability  distribution  around  some  type  of  point  estimate  (or  possibly  around 
a point  estimate  along  with  an  indication  of  the  distribution  variance).  It  is  something 
close  to  this  latter  approach  that  is  recommended  here  in  order  to  achieve  the  required 
simplicity  in  structuring  the  input  data. 

It  is  felt  that  a limited  choice  on  the  exact  form  of  the  distribution  to  be 
used  can  be  left  to  the  analyst  without  introducing  undue  complications.  The  recom- 
mended forms  are  either  the  uniform  (i.e.,  rectangular)  or  the  triangular  distribution. 
Either  of  these  can  be  easily  specified  by  indicating  the  mean  and  a range  around  the 
mean.  This  type  of  estimate  eliminates  the  need  to  formulate  a probability  statement 
to  describe  every  piece  of  input  data.  Data  items  known  with  complete  certainty  would 
be  input  as  a single  point  with  no  range  associated  with  them.  The  rectangular  distribution 
would  be  used  when  it  is  felt  that  no  information  whatsoever  is  available  about  the 
magnitude  of  the  estimated  or  future  cost  with  respect  to  its  estimated  range.  The 
triangular  distribution  lends  itself  to  the  situation  where  sufficient  information  is 
available  so  that  one  is  at:  least  able  to  say  that  the  occurrence  of  a cost  near  the  ex- 
treme end  of  the  estimated  range  is  less  likely  to  occur  than  a value  near  the  estimated 
expected  value.  It  is  logical  that  as  the  life  cycle  progresses  through  the  conceptual, 
validation  and  development  stages,  the  ability  to  refine  cost  estimates  will  increase. 

This  should  result  in  a general  narrowing  of  the  input  cost  ranges  as  well  as  a shift 
from  rectangular  distributions  to  triangular  distributions.  In  addition,  the  emergence 
of  primary  cost  drivers,  possibly  as  the  result  of  sensitivity  analyses,  will  occur  and 

emphasis  can  be  placed  on  refining  these  estimates  to  a higher  degree  than  the  less 

important  cost  estimates. 

Model  Output  Structure:  As  indicated  earlier,  the  prime  consideration  in  structuring 
the  output  is  to  make  i t easy  to  understand  and  thus  give  visibility  to  the  costs  (and 
the  associated  uncertainty  incurred  by  an  equipment  item  over  its  life. 

A basic  item  of  output  required  from  the  Monte  Carlo  analysis  is  a probability 
distribution  describing  the  life  cycle  cost  of  an  item.  This  could  appear  in  tabular 
form  or  as  shown  graphically  in  Figure  10  below. 

Once  this  is  obtained,  it  is  a relatively  easy  matter  to  give  the  degree  of 
confidence  associated  with  any  particular  interval  estimate  of  life  cycle  cost.  The 

example  in  Figure  10  shows  that  there  is  a <)07.  chance  that  the  total  life  cycle  cost  of 
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some  hypothetical  system  will  be  between  $1,200,000  and  $980,000.  A program  manager  is 
now  in  a much  better  position  to  make  decisions  concerning  program  funding  with  uncertain- 
ties concerning  cost  exposed  for  consideration. 


$980,000  $1,200,000 

Life  Cycle  Cost  in  Dollars 


FIG.  10  - DISTRIBUTION  OF  LIFE  CYCLE  COST 


It  may  be  desirable  to  structure  the  probability  distribution  of  life  cycle 
cost  in  terms  of  a cumulative  probability  distribution.  This  information  would  then 
indicate  the  probability  of  a life  cycle  cost  having  a specific  value  or  less.  It  could 
be  shown  graphically  as  in  Figure  11  below. 


$980,000  $1,200,000 

Life  Cycle  Cost  in  Dollars 

11  - CUMULATIVE  DISTRIBUTION  OF  LIFE  CYCLE  COST 
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In  addition,  it  is  felt  that  an  option  should  be  available  where  life  cycle 

costs  would  be  broken  down  into  annual  amounts.  This  would  provide  a cost  profile  that 

would  further  aid  decision-making  by  giving  visibility  to  the  continuing  as  well  as  the 
peak  year  funding  necessary  to  carry  out  a project.  The  penalty  paid  for  the  use  of 
this  option  would  be  an  increased  requirement  for  data  inputs.  Cost  data  would  either 
have  to  be  input  on  a year-by-year  basis  or  else  an  arbitrary  allocation  scheme  would 
have  to  be  exercised  by  the  model  to  accomplish  a year-by-year  assignment  of  cost. 

An  option  could  also  be  provided  so  that  costs  could  be  shown  either  with  or 
without  the  impact  of  inflation  and  the  application  of  a 10%  discount  factor. 

EXAMPLES  OF  LCC  MODEL  APPLICATION:  The  most  prominent  example  of  the  application  of  life 

cycle  cost  models  is  the  AN/ARC- 164  Procurement  Program.  The  program's  objective  was  to 
produce  a low  life  cycle  cost  radio  to  replace  a variety  of  radios  used  by  the  Air  Force, 
reducing  annual  maintenance  costs  and  eliminating  the  costs  of  supporting  more  than  one 
standard  radio. 
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An  LCC  model  was  provided  each  of  the  competing  contractors  who  were  to  use  their 
fixed  price  production  bid  as  the  acquisition  cost  and  estimate  the  O&M  costs  from  Air 
Force  equations  similar  to  those  presented  previously. 

The  achieved  LCC  was  to  be  verified  by  actual  field  measurements  of  the  LCC  parameters 
(e.g.,  reliability,  repair  costs,  data  costs,  etc.)  and  an  incentive/penalty  of  up  to 
25%  of  the  contract  price  determined  by  the  difference  between  the  computer  LCC  based  on 
field  measurements  and  the  LCC  based  on  the  contractor  estimates.  Figure  12  shows  the 
incentive/penalty  arrangement. 

One  other  provision  was  that  if  the  LCC  based  on  field  data  was  +3%  of  the  bid  LCC 
no  incentive  or  penalty  would  be  invoked.  This  was  to  allow  for  some  uncertainty  in 
the  results. 

The  most  significant  06<M  cost  driver  was  determined  to  be  the  reliability  of  the 
radio.  The  successful  contractor,  Magnavox,  Ft  Wayne,  Indiana,  made  his  own  analysis 
of  the  cost  vs  the  benefit  of  various  values  of  radio  mean  time  between  failures  (MTBF) 
sunrnarized  in  Figure  13.  On  this  basis,  he  bid  on  LCC  based  on  a 1,000  hour  MTBF.  He 
then  geared  his  reliability  program  to  achieve  that  figure  and  the  LCC  field  tests  in- 
dicate he  actually  did  better  than  anticipated.  The  final  LCC  computation  from  the 
field  results  was  within  3%  of  the  bid  LCC  and  hence,  no  incentive/penalty  was  actually 
paid. 


LCC  INCENTIVES  TO  MANUFACTURER 

POSITIVE  INCENTIVE  (%  OF  Ay) 


NEGATIVE  INCENTIVE  (%  OF  AT ) 


FIG.  12 
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This  program  thus  provided  a practical  exercise  of  LCC  modelling,  and  its  apparent 
success  in  achieving  its  LCC  goals  will  likely  lead  to  more  such  exercises. 

Another  application  of  LCC  modelling  is  in  the  use  of  Reliability  Improvement  War- 
ranties (RIWs)  where  LCC  modelling  is  used  to  determine  whether  a warranty  arrangment 
is  more  cost  effective  than  the  normal  situation  of  maintenance  by  the  purchasing  organ- 
ization. The  RIW  requires  the  producer  to  make  all  repairs  to  his  product  for  an  ex- 
tended period  of  time  (normally  3-5  years)  and  hence,  is  an  incentive  for  him  to  make 
all  possible  cost-effective  improvements  to  reliability  and  maintainability.  In  apply- 
ing the  warranty,  the  cost  of  the  warranty  and  its  benefits  are  weighed  against  the  cost 
of  organic  maintenance,  through  LCC  modelling. 

One  application  of  a RIW  was  to  the  AN/ARN-118  TACAN  procurement.  A life  cycle  cost 
analysis  determined  that  a four  year  warranty  followed  by  organic  maintenance  would  have 
a lower  LCC  than  pure  organic  maintenance  for  a ten  year  life  cycle.  14 

; •' 

The  contractor  also  committed  himself  to  an  MTBF  guarantee  of  800  hours.  This 
provision  required  the  supply  of  additional  spares  to  the  Air  Force  if  the  field  MTBF 
did  not  achieve  800  hours.  The  contractor  hence  had  an  incentive  to  achieve  high  reli- 
ability both  to  reduce  repair  costs  for  higher  profit  under  the  warranty  fee,  and  to 
avoid  the  penalties  of  not  meeting  the  MTBF  guarantee.  Latest  field  results  indicate  a 
MTBF  of  1800  hours  is  being  achieved.  The  success  of  this  program  will  undoubtably  spur 
the  further  use  of  warranties,  and  LCC  modelling  will  be  required  in  each  application  to 
weigh  the  life  cycle  costs  of  the  warranty  approach  against  those  of  the  conventional 
approach. 
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SUMMARY 


Improved  control  of  thoee  life-cycle  coots  that  are  associated  with  reliability 
requires  better  understanding  of  how  they  arise,  and  of  the  relationships  between 
the  many  possible  sorts  of  investment  in  better  reliability  and  maintainability 
and  the  benefits  it  achieves.  Basic  problems  in  deriving  these  relationships  are 
discussed,  with  particular  reference  to  overcoming  difficulties  of  data  acquisition, 
and  to  methodology  for  handling  such  complex  properties  as  the  "reliability"  and 
"maintainability"  of  airborne  weapons  systems.  Specific  problems  of  Cost  Analysis 
are  discussed,  including  some  novel  aspects  of  "bum- in".  The  need  is  stressed  for 
dealing  with  operational  effectiveness  on  a par  (and  Jointly)  with  cost  estimating, 
if  the  results  of  the  latter  can  be  expected  to  have  a significant  influence  on 
decision-making:  some  of  the  consequent  problems  of  analysis  are  considered.  In 
the  conduct  of  projects,  the  importance  is  stressed  of  providing  an  adequate 
information  system,  to  aid  timely  decisions  on  the  trade-offs  and  investments 
associated  with  reliability  and  maintainability. 


INTRODUCTION 

1.  It  is  first  necessary  to  explain  the  term  "reliability-associated  life-cycle  costs".  This  is  taken 
here  to  include  any  cost  incurred  in  achieving,  improving  and  accommodating  the  properties  of  "reliability" 
and  "maintainability"  that  are  experienced  in  service.  In  what  follows,  these  properties  will  be 
considered  jointly  (for  reasons  that  will  appear)  and  will  be  referred  to  as  "R  A M". 

2.  The  costs  discussed  in  this  paper  fall  into  two  main  categories: 

a.  RAM  investment  costs:  these  are  the  costs  attributable  to  activities  that  are  aimed  at 
achieving  or  improving  the  reliability  and  maintainability  of  systems.  Figure  1(a)  depicts  the 
phases  during  a life-cycle  in  which  these  activities  can  take  place.  It  is  to  be  noted  that  invest- 
ment can  continue  after  equipment  enters  service,  in  the  form  of  modifications  programmes  aimed  at 
improving  RAM. 

b.  Support  cOBia  associated  with  RAM:  these  sire  the  costs  of  maintenance  attributable  to 
"unreliability"  and  "unmaintainability"  that  is  experienced  during  the  service  life  of  the  system. 

Figure  1(b)  illustrates  the  sort  of  cash  flow  associated  with  these.  It  rises  sharply  during  the 
initial  provisioning  of  spares,  test-equipment  and  other  supporting  items,  shortly  before  entry  to 
service.  After  this  point  it  may  fluctuate  considerably,  for  various  reasons,  but  particularly  as 
the  amount  of  repair  work  or  scheduled  maintenance  changes  in  response  to  initial  learning  processes 
and  to  R A M modifications  programmes. 

3.  There  seems  to  be  considerable  scope  for  controlling  life-cycle  costs  through  savings  in  these  support 
costs,  since  they  make  an  important  contribution  to  overall  expenditure.  In  Reference  1,  for  instance,  a 
break-down  of  Royal  Air  Force  Costs  suggests  that  as  much  as  3C%  - 40%  of  the  annual  RAF  budget  might  be 
attributable  to  them. 

4.  Authorities  in  life-cycle  costing  have,  however,  also  stressed  (References  2 and  3.  for  example)  that 
the  potential  life-cycle  costs  of  airborne  weapons  systems  are  largely  determined  in  the  extreme  'front 
end'  of  projects  (ie  before  full-scale  development  has  begun),  particularly  in  the  conceptual  phase.  I 
do  not  propose  to  challenge  this,  but  would  point  out  that  provisions  to  meet  requirements  for  enhanced 
'performance'  capabilities  (in  relation  to  those  of  earlier  projects)  can  strongly  influence  the  initial 
design  options.  They  may  therefore  limit,  to  some  extent,  the  control  of  life-cycle  costs  that  can  be 
exercised  in  the  early  phases,  unless,  as  seems  unlikely,  large  extensions  to  delivery  time-scales  are 
amongst  the  acceptable  options. 

5.  Recent  studies  in  the  UK,  sponsored  jointly  by  the  Royal  Air  Force  and  the  Air  System  Controllerate 
of  Ministry  of  Defence  (Procurement  Executive)  have  explored  in  detail  the  costs  of  R A M investments, 
and  their  relationships  with  life-cycle  support  costs.  They  have  also  been  concerned  with  the  influence 
of  R A M investments  on  the  operational  effectiveness  of  aircraft  fleets,  and  with  the  effects  on  dates  of 
delivery  to  service  that  might  result  from  additional  RAM  activities  during  project  definition, 
development  and  production. 

6.  The  question  of  'operational  effectiveness'  will  be  discussed  in  more  detail  later,  but,  before 
proceeding,  some  clarification  is  needed.  Expressions  of  operational  effectiveness  can  take  the  following 
forms: 

a.  extent  of  training  programmes,  or  other  exercises,  that  can  be  undertaken  in  peacetime 

b.  potential  flight-line  availability  in  case  of  periods  of  tension  or  war 

c.  effectiveness,  given  full  serviceability  on  the  flight-line,  that  would  be  achieved  during 
operational  sorties  in  representative  war-time  scenarios. 


1 


1' 


Of  course,  a.  is  mainly  an  adjunct  to  c.t  in  that  it  determines  the  potential  fitness  of  air  and  ground 
crews  for  wartime  roles. 

7.  Life-cycle  support  costs,  and  particularly  those  attributable  to  R 4 M,  can  obviously  be  varied  in 
accordance  with  the  level  of  operational  effectiveness  that  is  sought,  or  permitted.  It  is  therefore 
necessary  to  establish  some  reference  level  of  effectiveness,  if  comparisons  of  life  cycle  costs  are  to  be 
useful.  The  influence  of  a.  upon  c.  is  not  easy  to  quantify,  and  a convenient  working  assumption  is  to  take 
the  level  of  a.  observed  in  recent  practice  to  be  adequate  (ie  to  assume  that  c.  would  not  be  significantly 
greater  if  a.  were  increased). 

8.  If  one  is  seeking  the  reliability  that  minimises  life-cycle  costs  at  some  fixed  level  of  operational 
effectiveness,  then  the  familiar  representation  of  the  optimising  process  needs  modifying  along  the  lines 
shown  in  Figure  2.  To  the  pair  of  curves  (I  and  II  in  Figure  2)  that  are  usually  shown  is  added  Curve  III, 
to  represent  the  cost  of  making  good  the  loss  of  effectiveness  that  is  due  to  unreliability.  Means  for 
this  were  considered  in  a paper  given  in  Lecture  Series  No  47(Reference  4)  and  included,  for  example,  such 
means  as  simply  increasing  the  size  of  the  aircraft  fleet.  It  is  seen  that,  when  the  'operational 
effectiveness'  factor  is  taken  into  account,  the  effect  is  to  increase  the  estimate  of  the  optimum  level 
of  reliability. 

9.  In  recent  studies,  we  have  simplified  matters  by  considering  investment  costs  and  benefits  in  relation 
to  levels  of  investment  cost,  and  of  R 4 M,  that  are  estimated  from  systematic  observations  of  equipment 
already  in  service.  The  "benefit"  is  expressed  in  terms  of  both  reduced  support  cost  and  increased 
operational  effectiveness. 

10.  There  are  two  ways  in  which  the  support  costs  associated  with  R 4 K may  be  controlled.  The  first  is 
through  changes  in  the  inherent  reliability  and  maintainability!  the  second  is  by  changes  to  maintenance 
practices,  so  that  the  costs  of  dealing  with  given  levels  of  R 4 M may  be  reduced.  In  the  recent  studies 
mentioned  earlier,  the  main  emphasis  has  been  on  the  former.  However,  during  the  course  of  the  studies 
some  indications  have  been  given  of  the  scope  for  reducing  costs  by  improving  maintenance  practices,  to 
be  compared  with  the  potential  savings  from  improving  the  R 4 M levels  themselves. 

STRATEGY  FOR  EXPLORATION  OF  INVESTMENT/RETURN  RELATIONS 

11.  The  prospects  for  drawing  useful  inferences  from  a purely  statistical  treatment  of  costs  in  past 
projects  on  airborne  systems  appear  to  be  very  poor.  The  R 4 M investments  and  returns  are  confounded 
with  many  other  important  factors  (including  the  prevailing  accountancy  procedures , and  the  particular 
levels  of  system  capability  or  technology  demanded). 

12.  One  is  therefore  forced  to  consider  the  detailed  mechanisms  by  which  investment  in  R 4 M activities 
has  influenced,  or  could  have  influenced,  support  costs  and  operational  effectiveness.  To  investigate  all 
such  mechanisms  over  a number  of  whole  airborne  weapons  systems  * would  be  an  enormous  task.  It  would 
also  fail  to  exploit  the  observation  that  some  of  their  sub-systems*  lend  themselves  much  more  readily 
than  others  to  such  quantitative  study,  and  that  some  contribute  very  much  more  than  others  to  support 
costs  or  to  loss  of  operational  effectiveness. 

13.  The  problem, then ,is  to  make  a suitable  selection  of  sub-systems  for  detailed,  "case-history", 
treatment.  The  criteria  foi  selection  are  likely  to  include: 

a.  A well-recorded  and  intelligible  programme  of  development 

b.  A well-recorded  history  of  in-service  "R  4 M"  modifications  and  their  implementation. 

c.  Technology  that  is  sufficiently  modem  to  be  representative  of  future  projects. 

d.  Accessible  data  on  R 4 M levels  experienced  in  service,  and  from  which  to  estimate  their 
influences  on  maintenance  costs  and  operational  effectiveness. 

These  are  not  as  easily  satisfied  as  might  be  imagined,  a priori,  and  in  practice  the  number  of  cases 
worth  considering  for  detailed  investigation  may  not  be  very  large.  The  selection  of  sub-systems  finally 
chosen  for  the  recent  UK  Studies  is  shown  in  Figure  3. 

14.  Following  selection  of  the  sub-systems,  a method  of  study  for  each  one  can  be  outlined  as  follows: 

A.  Collection  of  representative  samples  of  data  on  defect  rates,  associated  maintenance  activities 
and  unit  costs  of  the  activities. 

B.  Collection  of  samples  of  data  from  which  to  estimate  losses  of  operational  effectiveness. 

C.  Estimate  the  life  costs  of  maintenance  associated  with  identified  generic  types  of  cause. 

D.  Estimate  the  losses  of  operational  effectiveness  associated  with  generic  types  of  cause. 


* The  term  'airborne  weapons  system'  is  used  to  describe  a whole  aircraft  and  its  payload.  The  term 
'sub-system'  is  used  to  denote  a ftinctlonal  portion  of  this  whole.  Examples  of  sub-system  are  aii^- 
conditioning,  propulsion,  nav-attack,  communications  transceiver,  etc. 
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E.  Highlight  the  most  important  causes  of  maintenance  costs  and  of  loss  of  operational  effectiveness. 

F.  Conduct  historical  review  of  the  project  with  particular  reference  to  the  cost  and  effectiveness 
of  the  R £ M activities, and  to  other  factors  affecting  in-service  reliability. 

G.  Hypothesize  (and  assign  costs  to)  additional  S i H activities  that  might  have  been  profitably 
applied  at  various  stages  of  the  project. 

H.  Estimate  the  R 4 M gains  associated  with  these  additional  activities. 

I.  Translate  these  gains  in  R 4 M to  savings  in  life  maintenance  costs  and  increased  operational 
effectiveness. 

J.  Generalize  the  results  of  I to  cover  whole  airborne  weapon  systems,  and  indicate  potential  returns 
on  investment  in  future  projects. 

K.  Give  a paradigm  for  the  conduct  of  8 1 H activities  in  future  major  projects,  based  on  A to  J above. 

It  is  seen  that  steps  A through  I constitute  a detailed  "Case  Study”  of  each  sub-system,  and  that  this 
involves  a thorough  investigation  of  the  "background";  ie,  to  the  initial  operational  requirements,  time- 
scales,  contractual  arrangements,  and  revisions  to  specification;  etc.  Steps  G through  I concentrate  on 
the  potential  for  improvement  (one  measure  of  which  is  the  achievable  reduction  in  life-cycle  costa)  to 
guide  the  formulation  of  policies  for  future  projects. 

DIFFICULTIES  OF  DEALING  QUANTITATIVELY  WITH  R & M 

15.  Complexity  of  R & M properties;  The  conduct  of  the  Case  Studies  has  served  to  remind  us  that 
"R  & M"  of  modem  airborne  weapons  systems  (and  of  their  constituent  sub-systems)  are  in  reality  very 
complex  properties.  All  definitions  of  8 i H prove  more  or  less  unsatisfactory,  either  because  their 
application  is  aimed  at  too  restricted  a field,  or,  if  they  are  intended  for  general  use,  because  their 
constituent  terms  are  vague. 


16.  Consider,  for  instance,  the  definition  of  reliability  given  in  Reference  5,  which,  as  an  attempt  to 
describe  the  concept  briefly,  seems  at  least  as  good  as  any  other: 

"The  ability  of  an  item  to  perform  a required  function  under  stated  conditions  for  a stated  period  of 

time" . 

It  is  when  one  comes  to  quantify  'ability'  and  'time',  and  to  specify  'functions'  (and  what  constitutes 
satisfactory  functioning)  and  'conditions'  that  the  difficulties  begin.  Imagine,  for  instance,  the 
multiplicity  of  functions  of  even  one  sub-system  (eg  an  airborne  interception  radar  or  nav-attack  equipment) 
and  the  enormous  variety  of  event  sequences  that  can  be  demanded  by  the  operator  within  a time-segment,  and 
it  is  soon  appreciated  that  Reliability  of  a whole  tactical  aircraft  is  an  extremely  complex  property. 

17.  The  concept  of  maintainability  also  presents  difficulties.  Some  attempts  at  definition  address  it  in 
terms  of  the  time,  or  of  manpower  needed,  to  repair  defects.  It  becomes  clear,  however,  in  Case  Studies  of 
the  type  referred  to  above,  that  the  cost,  duration  and  frequency  of  scheduled  maintenance  should  also  be 
taken  into  account  when  defining  quantitative  measures  of  maintainability. 

18.  Inter-dependence  of  R & M:  The  scheduled  maintenance  implies  policies  for  assigning  fixed  intervals 
for  the  replacement  or  refurbishment  of  certain  components  or  sub-assemblies.  It  is  then  impossible  to 
treat  reliability  and  maintainability  independently  of  one  another.  Indeed,  rates  of  'defects'  or 
'failures'  are  often  controlled  by  means  of  these  'life-ing'  policies,  the  most  familiar  examples  perhaps 
being  found  in  jet  engine  components.  For  some  purposes,  scheduled  maintenance  can  be  treated  as  the  response 
to  patterns  of  regularly  occurring  defects.  Its  effects  on  life  support  costs,  for  example,  can  sometimes 

be  estimated  in  a similar  manner  to  those  of  randomly  occurring  defects.  Dealing  with  consequences  to 
operational  effectiveness,  however,  calls  for  a sharp  distinction  to  be  made  between  scheduled  and  un- 
scheduled activities.  Additional  expenditure  on  scheduled  maintenance  activities,  spares  holdings  etc  can 
provide  a means  for  achieving  increased  ^availability' . 

19.  R 4 M may  also  be  considered  to  be  interdependent  in  another  sense.  Engineering  design  that  is  aimed 
at  reducing  the  cost  or  duration  of  individual  maintenance  activities  (for  example,  by  making  sub-assemblies 
or  components  more  easily  accessible)  can  lead  to  higher  rates  of  defects.  Multiplicity  of  interconnections, 
in  the  interests  of  modularity,  has  sometimes  been  identified  as  a cause  of  loss  or  reliability. 

20.  Pitfalls:  It  is  common  practice  to  express  reliability  in  terms  of  a mean  interval  (eg  operating  hours, 
flying  hours,  number  of  sorties,  etc)  between  specified  events  (eg  failures,  defects,  maintenance  actions, 
replacements,  etc).  This  tends  to  give  the  impression  that  reliability  is  simple  to  dead  with,  which  may 
well  be  the  case  with,  say,  domestic  television  sets,  in  which  both  the  usage  and  the  forms  of  failure  are 
well  established  and  predictable.  With  airborne  weapons  systems,  however,  there  can  be  enormous  variety  in 
the  conditions  prevailing  during  an  "interval",  and  in  the  types  of  "events".  The  variations  in  the  conse- 
quences of  an  "event"  (as  measured  by  repair  cost,  or  loss  of  mission  effectiveness,  for  example)  can  be 
very  large.  This  is  why  "Reliability  Growth  Models"  that  deal  in  systematic  trends  in  mean  time  between 
failures  can  be  so  misleading,  since  the  residual  types  of  failure  may  have  disproportionately  serious 
effects  as  compared  with  those  that  have  been  eliminated. 

21.  Glow  emergence  of  R 4 M characteristics:  It  is  important  to  recognize  that,  at  the  start  of  a 
major  airborne  weapons  systems  project,  there  can  only  be  a vague  notion  of  the  more  significant  forms  of 
unreliability  that  are  going  to  reveal  themselves  during  development  and  production,  and,  subsequently,  in 
operational  service.  Which  is  to  say  that  specifications  of  Mean  Times  Between  Failures  cannot  serve  as 
much  more  than  a frame  of  reference  in  the  initial  phases  of  a project.  Data  that  are  potentially  accessible 
during  late  development,  particularly  after  flight-testing  has  commenced,  can  be  used  as  inputs  to  cost/ 
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manner.  The  specification  of  R 4 M objectives  during  a project  is  essentially  an  iterative  process.  They 
become  more  realistic  and  closely  defined  as  the  project  progresses.  Although  an  overall  MTBF,  with  a 
detailed  breakdown  between  sub-systems  should  be  assigned  initially,  the  counting  of  "failures"  and  what 
is  to  contribute  to  accumulated  "time"  is  negotiable  in  the  light  of  experience.  The  main  problem  is  to 
convert  the  flood  of  data  that  is  released,  or  potentially  available,  during  development,  production  and 
early  service  into  a timely  flow  of  feedback  information. 

22.  These  characteristics  of  the  R 4 M properties  of  complex  systems,  rule  out  the  more  obvious  approaches 
to  studying  the  lnvestment/retum  relationships.  For  example,  it  might  have  been  supposed  that  the  following 
method  could  be  followed: 

a.  Establish  the  relationship  between  investments  in  R 4 M,  and  the  changes  thereby  induced  in  rates 
of  defects,  rates  of  scheduled  maintenance  activities,  repair  times  etc. 

b.  Independently  of  a.  establish  the  effects  that  given  changes  in  the  rates  of  defects,  etc.  have,  or 

would  have,  on  support  costs  and  force  effectiveness. 

c.  Combine  the  relationships  of  a.  and  b.  so  as  to  derive  the  required  cost/benefit  relationships. 

In  practice,  such  a method  is  not  feasible,  for  the  basic  reason  that  it  is  not  until  one  has  investigated 
the  nature  of  the  defects,  scheduled  maintenance  actions  etc, that  one  knows  how  to  express  usefully  changes  in 
their  rates.  T~  ir  nature  is  understandable  only  when  both  the  following  properties  have  been  investigated!— 

a.  The  ultimate  influence  of  all  observed  types  of  defect,  and  scheduled  maintenance  actions  etc.  on 

support  costs  and  operational  effectiveness. 

b.  The  mechanisms  whereby  changes  in  engineering  practices  can  change  the  incidences  of  the  observed 

kinds  of  defect,  etc. 

It  is  therefore  apparent  that  an  iterative  approach  is  required,  in  which  the  appropriate  classifications 
of  alleged  defects  become  increasingly  evident  as  more  is  learned  of  their  basic  causes  and  effects. 

OTHER  BASIC  PROBLEMS 

2J.  Most  of  these  basic  problems  are  associated  in  some  way  with  difficulties  of  acquiring  the  necessary 
data.  To  some  extent  they  arise  because  the  recording  and  processing  of  data  suitable  for  R 4 M investment/ 
return  studies  has  not  been  a major  pre-occupation  of  routine  data  systems.  To  this  extent  they  can  be 
overcome  in  future  studies,  but  several  fundamental  problems  will  persist. 

24.  Before  considering  these  further,  it  is  well  to  remind  ourselves  of  some  of  the  main  items  of  R 4 M- 
associated  costs  that  have  to  be  taken  into  account.  These  are  set  out  in  Figure  4,  in  which  it  should 
first  be  noted  that  the  items  of  investment  cost  listed  in  Figure  4a  represent  a broad  classification.  In 
practice  they  have  to  be  broken  down  further  into  numerous  sub-divisions  to  correspond  to  the  many  forms 
that  R 4 M enhancement  activities  can  take.  There  are  general  problems  in  deriving  development  and  pro- 
duction costs,  and  their  availability  is  dependent  on  type  of  contract.  Attribution  to  their  R 4 M 
content  is  inherently  difficult,  because  R 4 M activities  tend  to  be  embedded  in  the  more  general  processes 
of  development  and  production. 

25.  Several  points  emerge  from  the  matrix  of  R 4 M-associated  support  cost  items  shown  in  Figure  4b.  The 
first  is  the  complexity  of  the  process  of  estimating  these  costs,  particularly  when  one  recalls  the  variety 
of  unscheduled  maintenance  actions  (eg  to  deal  with  many  types  of  genuine  defects,  of  alleged  defects  that 
are  not  confirmed,  and  of  accidental  damage  not  attributable  to  unreliability,  etc).  The  second  point  is 
that  it  would  be  very  remarkable  if  routine  data  systems  were  to  provide  directly  the  entries  to  elements 
of  the  matrix,  for  a particular  type  of  aircraft:  and  even  more  remarkable  if  these  were  directly  available 
for  sub-systems  of  the  particular  type  of  aircraft.  It  is  found  in  practice  that  the  processes  of 
estimating  these  entries  are  speculative  and  to  some  extent,  arbitrary,  largely  because  of  problems  of 
attribution.  One  has  to  proceed  through  indirect  measures  of  cost,  such  as  the  manning  levels  or  manhours 
attributed  to  particular  sub-systems.  Moreover,  it  is  necessary  to  take  into  account  items  of  'fixed  outlay' 
(in  addition  to  running  costs)  that  commit  expenditure  even  before  systems  have  entered  service.  The 
further  process  of  associating  support  costs  with  particular  kinds  of  defects  or  scheduled  maintenance 
activities,  (and  hence  to  specific  deficiencies  of  R 4 M for  given  sub-systems,  is  therefore  one  of  gross 
approximation.  The  process  is,  however,  well  worth  attempting,  because  useful  assessment  of  cost/benefit 
ratios  can  be  made  even  though  the  estimates  of  the  R 4 M-associated  support  costs  are  no  better  than  a 
binary  order  of  approximation. 

26.  Multifunctional  Organisations  Several  large  organisations  serve  as  sources  of  data  for  the  above 
estimates,  involving  manufacturers  and  procurement  agencies,  as  well  as  the  operators  of  the  weapons  systems. 
Difficulties  of  attribution  arise  because  these  organisations  are,  of  necessity,  concerned  with  a wider 
range  of  functions  than  is  implied  by  the  acquisition,  support  and  peacetime  operation  of  the  particular 
systems  under  study.  Manufacturers,  for  instance,  may  be  concerned  with  a much  wider  range  of  products, 

and  their  spare  parts  and  repair  facilities  may  serve  a variety  of  customers,  yet  be  subject  to  some  common 
overhead  costs.  Maintenance  personnel  of  the  armed  services  are  likely  to  have  potential  wartime  roles 
(eg  repair  of  battle  damage)  in  addition  to  peacetime  functions,  and  their  mannii^j  levels  are  likely  to 
reflect  the  several  functions.  For  this  reason  alone,  R 4 M lnvestment/retum  studies  must  be  based  on 
estimates  of  the  marginal  effects  of  changes  in  investment  and  demand.  Great  care  has  to  be  taken  when 
quoting  unit  costs,  particularly  with  respect  to  defining  the  reference  levels  of  cost/activity  upon 
which  they  axe  baaed.  The  need  is  also  implied  to  construct  from  past  experience  a picture  of  procurement 
and  operation  that  can  be  periodically  updated  to  take  account  of  general  trends  within  the  organisations 
concerned. 

27.  Operational  effectiveness  data.  Without  going  into  detail  at  this  stage,  it  should  be  pointed  out 
that  difficulties  of  data  acquisition  are  by  no  means  confined  to  those  associated  with  costing.  In 
particular,  the  estimation  of  loss  of  potential  wartime  effectiveness  by  aircraft  fleets,  due  to  R 4 M 
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shortcomings,  makes  demands  for  data  that  are  extremely  difficult  to  satisfy  through  routine  peacetime 
procedures  of  data  recording.  The  influences  of  defects  on  operational  effectiveness  are  no  less  varied 
than  their  influences  on  support  costs. 

20.  Basic  sampling  problem.  It  is  inevitable,  if  recent  (and,  therefore,  technologically  representative) 
system  acquisitions  are  to  be  studied,  that  samples  of  defect  rates,  and  other  rates  of  maintenance 
activities,  are  based  on  a relatively  short  segment  of  In-Service  life  (eg  upon  the  first  } years  of  a 
service  life  of  10-15  years).  Variations  of  such  rates  throughout  service  life  are  illustrated  in  Figure 
5.  It  is  to  be  expected  that,  for  any  one  sub-system,  several  plots  of  this  kind  would  be  required,  each 
corresponding  to  one  of  the  several  kinds  of  "significant"  defect  or  scheduled  maintenance  action  that  have 
emerged  in  service;  (ie  'significant'  with  respect  to  support  cost  or  to  operational  effectiveness).  One 
cannot  avoid  speculative  assumptions  on  future  trends.  One  can  ensure,  however,  that  in  comparing  returns 
on  R 4 M investment  to  be  expected  from  different  kinds  of  system,  or  sub-system,  that  the  assumptions  (eg 
on  learning  processes,  on  the  ameliorative  effects  of  modification  programmes,  and  on  'ageing*  processes) 
are  reasonably  consistent. 

WARNING  ON  LIFE-COST  MODELLING 

29.  The  foregoing  paragraphs  are  sufficient  to  alert  the  reader  to  the  dangers  of  embarking,  ab  initio,  on 
the  construction  of  elaborate  models  of  life— cycle  costing.  As  in  most  areas  of  operational  research,  the 
models  and  their  input  data  are  in  a 'chicken  and  egg’  relationship.  My  advice  is  to  take  a look  at  the 
available  sources  of  data  and  start  with  the  simplest  of  models,  then  reconsider  the  forms  of  data  that 
could  be  useful,  and  continue  iteratively.  It  seems  doubtful  whether  elaborate  models  would  ever  be 
Justified  in  the  study  of  R 4 M investment/retum  relations,  in  view  of  the  basic  assumptions  that  have  to 
be  made.  I am  inclined  to  extend  this  advice  to  all  forecasts  of  total  life-cycle  cost  of  high  performance 
airborne  weapons  systems,  on  the  grounds  that,  to  be  valid,  these  must  take  account  of  operational  effect- 
iveness levels  to  be  achieved  during  service  life,  and  that  the  prediction  of  these  will  always  remain 
imprecise. 

SPECIFIC  PROBLFMS  OF  DATA  ACQUISITION 

30.  Incidence  of  defects.  In  addition  to  the  inherent  variations  in  defect  rates  during  service  life 
(para  28,  above),  three  potential  sources  of  gross  errors  of  measurement  have  to  be  dealt  with: 

a.  Defects  of  a sub-system  may  be  wrongly  attributed  to  other  sub-systems  of  the  aircraft.  It  is 
therefore  important  to  define  unambiguously  the  sub-systems  under  study  and  to  ensure  that  the  system 
of  reporting  is  capable  of  associating  identified  defects  with  the  parent  sub-system,  particularly  when 
these  occur  in  components  of  a type  that  is  common  to  more  than  one  Bub— system. 

b.  The  operational  usage  of  the  sub-system  during  the  period  sampled  may  be  untypical.  For  avionics 
equipments,  in  particular,  the  use  of  Elapsed  Time  Indicators  is  helpful,  and  so  also  are  means  of 
collating  routine  stress  measurements  on  the  aircraft  (eg  from  "g"  meters). 

c.  Even  with  the  above  facilities,  there  is  the  risk  of  attributing  flying  hours  of  an  aircraft  fleet 
to  sub-systems  (particularly  avionics)  that  have  not  in  fact  been  carried,  or  operated,  on  all  flights. 
It  may  be  that  the  only  way  to  guard  against  this,  without  imposing  a tedious  burden  on  routine 
maintenance  reportingjis  by  a system  of  spot-checks  at  operational  stationB. 

31.  Assigning  causes  to  defects.  The  most  cost-effective  improvements  in  K 4 M policies  cannot  be  found 
unless  the  causes  of  defects  from  past  projects  can  be  identified.  This  is  by  no  means  as  straightforward 
as  might  be  thought,  because  the  reporting  of  defects  leaves  much  scope  for  interpretation.  It  seems  to  be 
common  experience  that  a substantial  proportion  of  defect  reports  is  ultimately  classified  as  'no  fault 
found'.  Where  a fault  is  confirmed,  it  may  be  attributable  to  mishandling  rather  than,  say,  to  engineering 
design,  or  faulty  workmanship.  Close  investigation  of  patterns  of  defects  reported  by  the  routine  data 
system  often  leads  to  a changed  diagnosis;  for  instance,  through  the  highlighting  of  "epidemics"  at  centres 
of  operation  or  maintenance,  or  of  associations  between  defects  and  particular  tail-numbers  in  the  aircraft 
fleet.  For  many  sub-ay sterna  (eg  complex  avionics  and  propulsion)  participation  by  the  respective  manu- 
facturers is  essential  to  useful  diagnosis. 

32.  Effect  of  bum- in  on  defect  patterns.  Since  burn-in  has  become  such  a common  practice,  particularly 
for  avionics  sub-systems,  it  is  of  great  interest  to  measure  its  effect  on  in-service  defect  patterns. 

There  is  a serious  lack  of  relevant  data  from  pest  projects.  This  can  be  remedied  in  future  by  controlled 
experiments  that  exploit  in-service  data,  and  collate  them  with  the  bum-in  histories  of  sets  of  similar 
equipments,  as  described  in  para  49,  below. 

33.  Spare  parts  consumption.  Special  provisions  may  be  needed  to  determine  the  quantities  of  consumable 
spare  parts  used,  for  the  sub-systems  studied,  during  a given  period  of  sampling.  Difficulties  are 
particularly  likely  where  there  are  sorts  of  spares  that  are  common  to  several  types  of  aircraft  or  to 
several  sub-systems  on  the  same  aircraft.  In  addition  to  these  considerations,  there  is  the  question  of 
'end  effects'  if  one  is  forced  to  use  indirect  measures  of  spares  consumption;  such  as  the  quantities  of 
spares  of  a given  sort,  that  are  ordered  (or  delivered)  from  time  to  time.  In  this  case  the  length  of 
sampling  period  should  be  sufficient  to  keep  the  quantization  errors  within  acceptable  limits.  Some 
organisations,  particularly  the  manufacturers  of  the  sub-systems,  may  not  have  a standard  practice  of 
accounting  separately  for  the  spares,  labour  and  overheads  attributable  to  repairs  and  reconditioning, 

in  which  case  further  special  provisions  may  be  needed,  for  study  purposes. 

34.  Manpower  consumption.  The  same  difficulties  of  attribution  that  apply  to  the  measurement  of  spare 
parts  consumption  apply  also  to  manpower  consumption.  Additionally,  there  is  the  problem  of  distinguishing 
between  the  manpower  used  on  reliability  - associated  activities  from  that  consumed  by  other  duties  of 
maintenance  personnel  (such  as  re-fuelling,  flight-line  aircraft  handling,  role-charge  etc).  The  most 
direct  means  of  estimating  manpower  is  from  the  records  of  manhours  attributed  to  particular  tasks  at  the 
several  lines  of  maintenance.  It  is  desirable  that  these  be  checked  by  assessments  of  the  manning  levels 
associated  with  particular  types  of  aircraft,  with  a breakdown,  where  possible,  applicable  to  the  sub- 


systems  being  studied. 

55.  Cuantltles  of  rotable  spares.  It  may  not  always  be  possible  to  distinguish  the  purchase  of  those 
equipments  (or  parts  thereof)  that  are  intended  as  the  basic  aircraft  fit  from  those  needed  as  'pipe-line' 
stock  to  support  repair  cycles.  The  latter  can  be  an  important  element  of  reliability-dependent  cost,  and, 
to  estimate  it  in  such  cases,  data  are  needed  on  effective  sizes  of  front-line  and  reserve  fleets. 

56.  Causes  of  consumption  of  spareB  and  manpower.  Mention  has  been  made  (paras  50  and  51)  of  the 
problems  of  counting  and  classifying  defects.  Even  when  these  are  overcome,  special  provisions  may  well 
be  needed  if  the  spares  and  manpower  consumed  in  a given  sampling  period  are  to  be  attributed  to  particular 
types  of  defect  and  of  scheduled  maintenance  actions.  For  maintenance  within  the  operator's  organisations, 
it  has  to  be  decided  whether  more  detailed  inputs  to  a centralized  maintenance  data  system  would  be 
justified  on  a routine  basis,  or  whether  ad  hoc  methods  of  spot-sampling  would  be  preferable.  Repair  and 
reconditioning  work  by  manufacturers  poses  more  difficult  problems  of  attribution  when  sub-contractors  sire 
also  involved.  In  addition  to  distinguishing  between  scheduled  and  unscheduled  maintenance  there  may  also 
be  the  problem  of  separating  out  work  and  materials  expended  on  in-service  modifications.  Within  the  total 
quantities  of  spares  and  manpower  consumed  by  the  modifications  programmes,  data  will  be  needed  to  dis- 
tinguish the  contribution  to  those  modifications  aimed  at  Improving  R & M,  and  to  enable  this  to  be 
attributable  at  sub-system  level.  Such  data  are  likely  to  be  more  readily  available  for  the  avionics  sub- 
systems than  those  parts  of  the  aircraft  that  are  commonly  dealt  with  by  'blanket'  provision  of  modifications 
funds. 

37.  Operational  implications  of  R A M.  The  three  aspects  of  operational  effectiveness  given  in  paragraph  6 
suggest  the  kinds  of  data  needed.  Routine  maintenance  data  systems  may  tend  to  overemphasise  the  effects  of 
defects  on  peacetime  training  activities,  at  the  expense  of  necessary  detail  on  potential  loss  of  wartime 
effectiveness.  In  view  of  the  many  possible  wartime  scenarios,  even  for  a fleet  of  aircraft  of  the  same 
type,  collection  of  the  latter  sort  of  data  could  impose  a heavy  burden  on  maintenance  staff,  if  undertaken 
on  a routine  basis. 

58.  For  the  Implications  of  defects  during  the  sorties  in  which  they  arise,  a functional  classification 
is  needed,  suitable  for  inputs  to  operational  models  of  combat,  support,  strike  or  reconnaisance  activities. 

At  least,  these  should  distinguish  those  defects  that  curtail  the  sortie,  but  it  is  also  important  to  make 
finer  distinctions  in  degrees  of  loss  of  effectiveness.  For  example,  in  the  case  of  radar  equipment  for 
airborne  interception,  the  incidences  of  losses  of  the  various  modes  should  be  distinguishable  in  the  data, 
and  reductions  in  ranges  of  surveillance  and  missile  control  should  also  be  recorded.  It  may  well  be  that  the 
collection  of  this  sort  of  data  would  require  special  visits  by  analysts  to  operational  stations,  and, 
during  these,  exceptional  contributions  from  air  crews  and  maintenance  personnel.  These  could  be  used  to 
check,  and  to  complement,  the  day-to-day  system  of  data  collection. 

59.  Flight-line  availability  at  operational  stations  depends  upon  both  scheduled  and  unscheduled 
maintenance  activities.  Commonly  available  data,  such  as  the  proportion  of  aircraft  ready  to  fly  at  some 
stated  time  of  day,  are  of  limited  value  for  application  to  wartime  scenarios.  They  need  to  be  supplemented 
with  data  on  the  degrees  of  unserviceability  (giving  causes)  and,  in  particular,  on  recovery  times.  Data 
for  the  latter  should  be  appropriate  to  assessing  the  effect  of  alternative  assumptions  on  maintenance 
resources  (stocks  and  manning  levels)  and  on  other  features  of  battle  conditions,  such  as  damage  and 
attrition. 

SOME  PROBLEMS  OF  COST  ANALYSIS 

40.  Effects  of  changes  in  R A M on  maintenance  demand.  The  effects  of  improvements  in  R A M (eg  those  that 
could  have  been  achieved  with  additional  investments  during  development  or  production)  will  be  to  modify 
the  shapes  of  the  event-rate  curves  shown  in  Figure  5. 

41.  For  each  classified  type  of  defect  or  scheduled  maintenance  action,  new  curves  will  be  produced, 
reflecting  decreases  in  the  quantities  of  consumable  spare  parts, and  in  the  labour  required, for  repairs  or 
reconditioning.  The  difference  between  the  areas  under,  respectively,  the  original  and  the  new  curves 
gives  the  reduction  in  the  particular  type  of  running  (or  on-going)  demand  on  maintenance  resources:  this 
is  illustrated  in  Figure  6.  In  practice,  due  to  the  difficulties  enumerated  earlier,  one  may  have  to 
settle  for  gross  simplifications  to  these  curves,  using,  for  instance,  estimates  of  life  average  rates 
(using  only  a very  broad  classification  of  types  of  defect)  based  on  samples  from  a short  segment  of  the 
life  cycle. 

42.  Estimation  the  effects  of  improved  R A M on  the  "fixed  outlay"  on  rotable  spares  and  on  holdings  of 
special  test  equipment  etc  .present  some  difficult  questions.  To  what  extent,  for  example,  can  the 
improvements  in  R A M be  predicted  at  the  time  at  which  decisions  on  initial  provisioning  have  to  be  made? 
However,  let  us  proceed  on  the  assumption  that  this  improvement  can  be  predicted  exactly.  It  would  then 
seem  reasonable  to  assume  that  the  fixed  outlay  would  vary  in  proportion  to  the  maximum  demand  during  the 
service  life  of  the  sub-system  being  studied: 

STOCK  OC  Max  (F(t)  A(  t) ) ; 
t 

where  F(T)  represents  the  quantity  of  flying  per  unit  calendar  time  during  service  life 

where  A(T)  represents  the  number  of  defects  or  scheduled  maintenance  actions  per  unit  flying  time. 

It  is  seen  that  knowledge  of  F(T)  depends  on  data,  or  assumptions,  on  the  rate  of  build-up  of  the  fleet,  or 
of  introduction  of  the  equipment  studied.  (There  are  also  such  finer  points  as  to  whether  the  stock  has 
any  end-of-life  value  and  whether  some  of  it  could  be  turned  into  consumable  spares). 
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45.  If  the  actual  case  history  shows  that  the  initial  provisioning  levels  of  rotables  and  test-equipment 
was  unduly  high,  then  it  may  necessary  to  postulate  a rationalized  procedure  (applicable  to  provisioning 


of  these  items  in  future  projects)  before  applying  the  proportionality  given  in  para  42 


44.  Relating  savings  to  reductions  in  demand.  We  have  considered  the  estimation  of  potential  reductions 

in  demands  on  resources  (spare  parts,  materials  and  manpower  for  repair  and  reconditioning),  but  not  yet  the 
ultimate  savings  in  support  costs.  The  difficulties  in  estimating  the  latter  stem  partly  from  the  dis- 
continuities (and  other  non-linearities)  in  'overhead'  costs,  and  partly  from  the  multifunctional  nature, 
discussed  earlier,  of  organisations  concerned  with  maintenance. 

45.  The  direct  costs  of  maintenance  manpower  (as  indicated  by  wage  rates,  for  instance)  are  small  in 
comparison  with  the  total  costs  that  might  be  attributed  to  it,  if  existing  budgetary  estimates  were 
allocated  pro-rata  to  the  sub-systems  studied.  They  remain  comparatively  small  even  after  the  non- 
reliability-dependent element  has  been  removed).  The  'overheads'  include  supporting  labour  as  well  as  the 
capital  facilities.  The  studies  of  potential  effects  of  additional  investment  in  R & M enhancement  suggest 
the  possibility  of  substantial  inroads  (eg  of  the  order  of  50%)  into  the  peacetime  maintenance  workload, 

in  certain  areas  of  technology.  The  potential  inroads  differ  considerably  for  the  respective  areas  of 
technology  however,  and  it  is  not  easy  to  arrive  at  a broad  generalisation.  However,  some  success  has  been 
achieved  in  particular  areas,  and  Figure  7 illustrates  the  sort  of  relationship  found  in  an  example  for 
which  peacetime  maintenance  was  the  sole  function  of  the  organisation  concerned.  The  discontinuities 
represent  such  discrete  processes  as  the  closure  of  whole  buildings  or  other  capital  facilities. 

46.  A much  more  difficult  problem  arises  when  considering  the  scope  for  reducing  manpower  and  its 
associated  overheads  at  operational  stations.  One  way  of  proceeding  would  be  to  express  the  potential 
benefit  in  terms  of  the  extra  effort  that  would  be  released  for  other  activities  (eg  repair  of  battle 
damage)  in  wartime,  and  hence  to  convert  to  units  of  operational  effectiveness  rather  than  to  those  of 
support  cost. 

47.  Allowing  for  differences  in  scales  of  procurement.  The  costs  of  investment  and  of  support  both  depend 
upon  the  quantities  of  aircraft  and  equipment  that  are  produced  and  operated.  Comparison  between  the  returns 
from  investment  in  the  R & M of  the  sub-systems  studied  require  some  kind  of  standardization  of  procurement 
scale,  and  a basis  for  reading  them  across  to  scales  in  future  projects.  Useful  approximations  can  be  made 

by  dividing  the  costs  into  three  classes,  according  to  dependence  on  numbers  produced  and  operated: 

a.  fixed  (eg  development  costs) 

b.  semi-fixed  (eg  test-chamber  facilities  for  quality  control  in  production) 

c.  pro-rata  (eg  spares  consumption  in  service,  or  in  the  repair  of  failures  in  bum-in) 

48.  Cost  effectiveness  of  bum-in.  Bum-in  gives  rise  to  reliability  investment  costs  that  comprise 
fixed,  serai-fixed,  and  pro-rata  elements.  The  pro-rata  and  semi-fixed  costs  tend  to  predominate  and,  no 
matter  how  lar* ® the  production  quantities,  a substantial  unit  cost  persists.  For  avionics,  a typical 
order  of  bum-in  cost  per  defect  detected  (in  test-chambers),  and  rectified  before  delivery,  is  1000  US 
dollars.  Increasing  the  duration  of  bum-in  tends  to  increase  this  unit  cost,  and  to  call  into  question 
whether  a cost/effective  basis  has  been  established.  Figure  8 shows  the  sort  of  functional  relationship 
inferred  in  the  case  of  an  avionic  equipment,  for  which  sub-samples  of  the  production  run  were  bumed-in 
for  much  longer  than  the  average.  A physical  interpretation  of  this  sigmoid  shape  is  that  the  initial 
slope  is  predominantly  due  to  the  'infant  mortality'  type  of  failure  that  bum- in  is  expressly  aimed  to 
eliminate.  The  rate  subsequently  settles  down  to  a level  suggested  by  the  slope  of  the  tangent  to  the 
point  of  inflection  (Fig  fl),  which  is  independent  of  infant  mortality  and  may  be  taken  to  have  been 
constant  up  to  this  point;  ie,  to  correspond  to  some  underlying  defect  rate  that  is  independent  of  infant 
mortality.  Further  time  in  the  test  chamber  leads  to  the  premature  ageing  suggested  by  the  curve  to  the 
right  of  the  point  of  inflection.  The  risk  of  proceeding  too  far  beyond  the  point  of  diminishing  returns 
is  obvious. 

49.  In  a thorough  cost/benefit  analysis  of  bum- in,  it  would  first  of  all  be  necessary  to  estimate  the 
extent  to  which  the  patterns  of  defects  detected  would  have  been  reproduced  in  service  (particularly, 
whether  they  would  mostly  arise  early  in  service  life).  Steps  to  collect  the  necessary  data  to  support 
such  estimates  have  been  suggested  above.  This  done,  and  the  consequent  savings  in-service  repair  costs 
estimated,  the  side-benefits  of  bum- in  would  also  have  to  be  taken  into  account.  These  include: 

a.  The  provision  of  information  on  systematic  defects,  which  can  give  a basis  to  programmes  of 
reliability  growth  in  production  (it  is  important  to  distinguish  this  from  the  elimination  of 
infant  mortality). 

b.  Review  of  a means  (for  comparison  with  other  methods,  such  as  batch-sampling)  of  sustaining  the 
contractor's  attention  throughout  the  production  process. 

, » 

c.  Any  side-benefits  that  arise  from  preventing  bursts  of  defects  in  early  service.  These  might 
include  the  prevention  of  loss  of  morale,  of  excessive  handling  (hence  of  failures  due  to  mishandling) 
and  of  hasty  re-provisioning  of  maintenance  resources. 

50.  Discounted  cash  flow  (DCF):  It  is  to  be  noted  that  investment  to  improve  RAM  can  be  made  at  any 
point  in  the  life  cycle,  but  that  those  made  in  the  earlier  phases  may  not  reap  their  full  benefit,  in  support 
cos t savings,  until  some  20  years  later.  In  a climate  of  high  interest  rates,  the  practice  of  discounting 
these  savings  can  have  a profound  effect  on  investment/retum  ratios. 

51.  There  does  not  seem  to  be  a generally  agreed  rationale  for  applying  DCF  adjustments  to  defence  project 
funding.  It  can  be  noted,  however,  that,  with  the  patterns  of  reliability-associated  cash  flow  that  can 

be  expected  over  the  life  of  an  aircraft  project,  and  with  an  annual  discount  rate  of  10°4,  an  undiscounted 
return  of  between  lj  to  1 and  2j  to  1 would  be  required  to  break  even  (in  a commercial  sense)  on  investment 
made  throughout  the  development  and  production  phases. 


v« 

52.  It  la  also  to  be  observed  that,  In  the  application  of  DCF,  It  might  be  possible  to  treat  investment 
in  bum-ln  as  a special  case,  with  considerable  simplification.  If  the  main  benefit  of  bum-in  is  in 
fact, to  eliminate  defects  in  early  service,  then  one  would  expect  to  reap  it  only  a short  while  after 
the  corresponding  Investment  occurs  on  the  production  line.  In  this  case,  the  effect  of  discounting 
might  be  negligible. 

ANALYSIS  OF  OPERATIONAL  EFFECT IVOJESS 

55.  We  touch  now  on  one  of  the  most  difficult  subjects  related  to  the  control  of  life  cycle  costs:  the 
analysis  of  operational  effectiveness  (paragraphs  6-8  above).  The  particular  aspect  of  concern  in  this 
paper  is  the  loss  of  effectiveness  due  to  unreliability  and  unmaintainability,  and  the  potential  for 
reducing  this  loss  by  improving  RAM.  In  the  comparison,  between  sub-system,  of  the  potential  returns 
they  offer  on  R A M investment,  the  two  forms  of  benefit  (viz:  reduced  support  cost  and  increased 
operational  effectiveness)  can  suggest  very  different  priorities  if  considered  in  isolation  from  one 
another.  Estimates  of  both  are  essential  to  sound  decision-making.  For  instance,  a recent  cost/benefit 
study  of  a proposed  reliability-»»provement  programme,  for  an  avionics  equipment  already  in  service, 
showed  a poor  return  in  the  torm  of  support  cost  savings.  This  was  because  much  of  the  life  support  cost 
had  already  been  committed  to  the  provision  of  rotable  spares  and  special  test  equipment.  There  was 
sufficient  fleet  life  remaining,  however,  for  the  expected  gain  in  operational  effectiveness  (equivalent 
to  adding  several  aircraft  to  the  fleet)  to  be  the  more  significant  consideration. 

54.  Need  for  high-level  measures.  The  measures  of  operational  effectiveness  should  lend  themselves  to 
selection  of  priorities  (between  sub-system  investments)  and  to  being  weighed  alongside  predictions  of 
support  cost  savings.  This  implies  taking  into  account  the  role  for  which  the  whole  airborne  weapons 
system  has  been  procured.  There  are  usually  several  alternative  ways  (in  addition  to  improving  RAM) 
of  off-setting  a loss  in  the  operational  effectiveness  of  an  aircraft  fleet,  and,  ideally,  one  should 
compare  them  all.  In  practice,  there  is  seldom  sufficient  data  for  this  ideal  procedure,  but  at  least  a 
useful  upper-limit  estimate  can  often  be  made  of  the  value  of  the  effectiveness  gain,  by  translating  it 
into  an  equivalent  change  in  the  number  of  aircraft  of  the  fleet,  and  hence  to  a life-cost  change.  In 
Reference  4,  Baker  has  considered  the  alternatives  of  fleet  expansion  and  more  intensive  flying  at 
constant  fleet  size.  His  arguments  suggest  that  of  the  two  alternatives,  the  cost  of  the  former  is 
likely  to  give  the  more  valid  measure,  and  that,  in  any  case,  their  costs  will  not  differ  greatly  unless 
there  has  been  a substantial  departure  from  the  optimum  allocation  to  aircraft,  men,  and  other  resources. 

55.  Operational  modelling.  This  will  usually  require  models  of  the  operation  of  the  airborne  weapons 
systems  in  their  wartime  roles.  In  many  cases,  the  estimation  of  high-level  measures  of  changes  in 
potential  wartime  effectiveness  will  be  hampered  by  shortage  of  input  data  (as  discussed  earlier). 

Represenataion  of  the  effects  of  battle  damage  to  aircraft  and  airfields,  and  of  supplies  of  maintenance 
resources,  for  instance,  will  be  subject  to  many  speculative  assumptions.  Elaborate,  general  purpose 
modelling  does  not  seem  appropriate  to  this  field.  The  data  that  are  readily  available  from  peacetime 
operations  Justify  only  the  most  rudimentary  models  initially.  These  should  be  used  in  sensitivity 
analyses  to  indicate  the  sorts  of  information  that  will  be  needed  from  special  processes  of  data  collection 
at  operational  stations. 

EXAMPLE  OF  WARTIME  EFFECTIVENESS  MODELLING 

56.  The  example  that  follows  is  in  illustration  of  the  initial  modelling  needed  for  a given  type  of 
airborne  weapons  system,  in  order  to  investigate  sensitivities  to  parametric  values  and  to  the  assumptions 
used.  It  represents  the  sort  of  model  that  can  be  suggested  by  a quick  look  at  those  data  that  are  readily 
available  in  peacetime. 

57.  Tactical  airforce  operations.  The  airborne  weapons  system  chosen  as  an  example  consists  of  an 
aircraft  and  payload  designed  for  the  close-support  role  in  land  battles.  We  make  use  of  the  basic 
assumption,  suggested  in  paragraph  7,  to  the  effect  that  the  levels  of  unreliability  in  peacetime  have 
not  been  such  as  to  interfere  significally  with  the  training  of  crews,  and  hence  have  not  affected  their 
wartime  preparedness. 

58.  The  pattern  of  close  support  is  represented  as  a demand  on  each  aircraft  for  bursts  of  intense 
flying  activities.  Each  of  these  consist  of  a sequence  of  sorties  that  are  spaced  as  closely  as  possible 
in  time.  Each  sequence  is  seen  as  a response  to  some  critical  event  in  the  land  battle,  such  as  an 
armoured  breakthrough  or  other  suitable  target  concentration.  The  sequences  are  taken  to  be  spaced  at 
comparatively  large  intervals  of  time,  so  that  the  sorts  of  maintenance  activities  that  can  be  undertaken 
within  a sortie  turnround  are  very  different  from  those  that  are  possible  between  sequences.  The  number 
of  sorties  in  a sequence  is  treated  parametrically. 

59.  Measure  of  effectiveness.  The  function  cf  a tactical  air  force  is  assumed  to  be  to  destroy  surface  i 

targets,  of  a small  number  of  generic  types  (eg  tanks).  The  effectiveness  is  taken  to  vary  in  proportion 

to  the  total  number  of  such  targets  destroyed  by  the  force  during  a battle.  In  these  terms,  loss  of 
effectiveness  will  depend  upon  both. 

< 

a.  attrition  of  the  air  force,  or  other  damage,  inflicted  by  enemy  action, 

b.  unavailability  of  aircraft  or  equipment  due  to  scheduled  maintenance  activities. 

c.  incidence  of  defects  due  to  unreliability. 

To  a first  order  approximation  it  is  reasonable  to  assume  that  the  occurrences  of  (a)  are  independent  of 
those  of  (c).  Their  effects,  however,  may  well  interact,  but,  as  an  initial  assumption,  throe  are  also 
taken  to  be  independent  of  one  another.  A further  simplification  to  the  analysis  can  be  made  by 
transferring  the  effect  of  (b)  to  support  costs:  ie  by  allowing  for  the  extra  resources  (eg  rotable  spares) 
to  be  provided  to  ensure  that  scheduled  activities  have  a negligible  impact  in  comparison  with  randomly 
occurring  defects.  In  the  cases  studied,  the  consequent  increase  in  life  support  cost  would  be  only  a 
small  percentage.  The  effectiveness, E,  is  therefore  taken  as  the  following  ratio: 


; 


E = targe •- n destroyed  at  a given  level  of  B i K 
targets  destroyed  given  perfect  reliability 


- - - equation  1. 

6C.  All  defects  observed  on  the  individual  sub-systems  of  the  aircraft  are  grouped  into  the  following 
three  classes: 


Type  1.  Defects  that,  in  wartime,  would  prevent  take-off,  or,  if  occurring  whilst  airborne,  would 
require  an  immediate  return  to  base. 

Type  2.  Defects  that  would  not  prevent  take-off  in  wartime  but  would  degrade  the  ability  to  destroy 
targets. 

Type  3.  Defects  that  have  no  influence  on  mission  capability. 

It  is  assumed  that  repairs  of  all  Types  1 and  2 defects  can  be  made  between  sequences  but  that  none  can  be 
made  in  the  intervals  between  sorties.  As  an  example,  a 3-sortiej per-sequence  operation  is  shown  in  Figure  9. 


*1.  Further  assumptions. 

(i)  Whenever  a Type  1 defect  occurs  in  a sequence,  the  aircraft  will  remain  out  of  action  until  the 
end  of  that  sequence. 

(ii)  Duration  of  weapons  release  period  is  small  compared  with  sortie  length,  and  occurs  mid-way  through 
the  sortie,  as  shown  in  Figure  9*  (This  simplifying  assumption  is  not  critical.  Provided  total 
weapon  release  time  remains  small,  releases  could  be  treated  as  though  spread  across  the  Borties, 
without  substantially  changing  the  results  from  the  model). 

(iii) On  each  occurrence  of  a Type  2 defect  on  a particular  aircraft  a reduction  coefficient,  at  , is 
applied  to  the  expected  number  of  targets  destroyed  by  its  next  weapons  release.  (This  coefficient 

is  used  to  represent  the  average  degradation  produced  by  all  types  of  defect.  It  is  treated  parametric- 
ally). 

(iv)  Turn-round  times  allowed  between  sorties  are  too  short  for  repair  of  Type  2 defects.  These  are 
therefore  carried  over  from  sortie  to  sortie,  within  a sequence.  If  a number,  m,  of  Type  2 defects 
has  accumulated  before  a weapon  release  period, the  factor  at  is  therefore  applied  to  the  expected 
number  of  kills  during  that  period. 

(v)  Types  1 and  2 defects  are  fully  repaired  in  the  intervals  between  sequences. 

(vi)  Types  1 and  2 defects  occur  at  random  during  the  sorties,  independently  of  one  another. 

(vii) At  the  beginning  of  the  first  sequence  of  the  battle,  there  are  no  aircraft  unserviceable,  due 
to  unreliability. 

62.  Derivation  of  the  effectiveness  ratio  (E).  This  follows  from  the  foregoing  assumptions,  using  the 
following  notation: 

X ^ = expected  number  of  Type  1 defects  per  sortie 
X „ ii  it  ti  ii  2 " *'  ** 

«=  degradation  factor  (para  59  (iii)  and  (iv)) 


n = number  of  sorties  in  a sequence  . 

Of  those .aircraft  flyable  at  the  start  of  a sequence,  the  proportion  reaching  the  point  of  weapons  release 
in  the  sortie  is  given  by 


«xp  £(j-£)\7 


Pm  such  aircraft,  the  effect  of  Type  2 defects  on  the  expected  number  of  targets  destroyed  during  the 
J sortie  is  given  by  multiplying  by  the  following  factor 


/\  * <*(j-£)\2  + «2(j-^)2xf  ♦ - - -j  exp yy 


Thus,  each  unit  expectation  at 
to: 


exp 


^ie  weapon  release  has  been  reduced  by  the  Type  1 and  Type  2 defect  rates 


♦Az  u->7} 

Substituting  x for  * Aj  *\2  (l-o^and  summing  over  all 
equation  1 that  *“ 

E = i |^exp  ^(2j-l)x7  - equation  • 

E = (l-e  2n*)/2n  SINHx  equation  } 


sorties  in  the  sequence,  it  follows  from 


i 


63.  The  estimated  loss  of  effectiveness  due  to  unreliability  is  given  by  100(l-E)%,  and  is  plotted  against 


Type  1 defect  rate,  A,,  tn  Figure  10,  in  which  the  effects  of  varying  the  degradat ion  factor,  « , the 
sequence  length,  n,  and  the  ratio  \0  \ ^ are  also  shown. 

64.  Further  development  of  the  model  (and  additional  data  required)  would  be  influenced  to  some  extent  by 
the  ranges  of  and  Ag  thatare  expected  to  be  of  interest.  Apart  from  this,  the  relative  insensitivity  to 
at,  over  quite  a wide  range,  suggests  that  over-elaboration  should  be  particularly  avoided  when  the  time 
comes  to  represent  the  degradations  due  to  individual  sub-systems,  (eg  when  replacing  at  by  a set  of 
distributed  variables).  The  effects  of  changing  the  length  of  sequence  are  large  enough  to  suggest  the 
need  to  take  a closer  look  at  repair  times,  perhaps  introducing  a set  of  distributed  variables  to 
represent  them  in  the  next  version  of  the  model. 

DEALING  WITH  PROPOSED  EXTENSIONS  TO  TIME  SCALE 

65.  Perhaps  the  most  Intractable  problem  encountered  in  studies  of  H 4 M investments  and  return* is  how  to 
put  a value  on  delays  to  in-service  dates.  The  question  arises  because  some  of  the  means  of  H A M 
enhancement  that  suggest  themselves  in  development  or  early  production  require  additional  time,  despite  a 
more  intense  use  of  other  resources.  On  what  can  an  operator  of  airborne  weapons  system  base  a decision 
when  informed  that  if,  for  instance,  lie  waits  another  year  for  some  new  equipment,  it  will  save  x million 
dollars  of  support  costs  and  give  a y*'.-  increase  in  operational  effectiveness  when  it  gets  into  service? 

66.  It  seems  reasonable  to  start  from  the  assumption  that  the  delay  would  require  an  extension  of  life 
(and,  perhaps,  additional  quantities)  of  obsolescent  equipment,  and  that  this  would  perform  the  required 
functions  at  a higher  cost  during  the  period  of  delay  than  would  the  new  equipment.  In  the  case  of  an 
airborne  weapons  system,  an  estimate  of  this  cost  difference  might  help  the  decision-maker,  but  seems  to 
be  inaccessible.  I believe  this  is  because  the  underlying  trends  in  the  cost/effectiveness  of  weapons 
system* tend  to  be  masked, mainly  by  changes  in  the  description  of  the  threats  that  they  are  intended  to  meet. 

67.  An  example  of  the  sort  of  trend  that  might  be  uncovered  is  given  by  the  costs  of  carrying  unit  weight 
of  payload  in  air  transport  operations  which,  it  can  be  argued,  serves  to  'isolate*  savings  that  are  almost 
wholly  due  to  the  'march  of  technology*.  The  data  given  in  Reference  6,  for  civil  air  transport,  are  shown 
in  Figure  11.  They  suggest  that,  by  a more  or  less  continuous  process,  (with  a alight  discontinuity 
following  the  introduction  of  Jet  aircraft  in  1956),  the  effectiveness/cost  ratio  was  doubled  during  the 

20  years  period  up  to  1973.  Each  of  the  major  components  of  cost  showed  a similar  trend. 

68.  It  is  for  consideration  whether  the  corresponding  trends  are  accessible  in  the  case  of  airborne 
weapons  system  (eg  in  anti-tank  capability  per  unit  cost)  and,  if  so,  what  techniques  should  be  used  to 
convert  them  into  estimates  of  equivalent  cost  penalties  for  given  delays  in  the  introduction  of  new  systems. 

ANALYTICAL  SUPPORT  FOR  THE  CONDUCT  OF  PROJECTS 

69.  The  techniques  by  which  investments  in  activities  for  improving  8 4 H are  brought  to  bear  throughout 
projects  have  been  enumerated  elsewhere.  The  UK  Ministry  of  Defence  publication  at  Reference  6 deals  with 
integrated  management  of  R 4 M in  avionics  projects  from  the  initiation  of  the  concept  through  to  maturity 
in  service.  In  the  USA,  Military  Standards  of  the  785  series  have  given  precepts  for  the  management  of 
RAM  that  are  applicable  at  the  level  of  whole  airborne  weapons  systems. 

70.  Trade-off's  play  a very  important  part  in  the  management  of  H A M activities.  For  example,  preliminary 
studies  can  indicate  the  potential  for  trading  reliability  and  maintainability  with  'performance'  variables, 
or  with  time-scale.  These  must  draw  heavily  on  systematically  exploited  'feedback'  from  earlier  projects, 
but  can  be  up-dated  as  better  data  become  available  during  the  conduct  of  the  project.  The  accommodation 

of  several  kinds  of  avionics  equipments  in  the  confined  space  and  usually  'difficult'  environment  of  the 
aircraft  ia  .mother  area  for  trade-off.  Selection  of  priorities  for  funding  additional  R 4 M activities 
is  required  throughout  a weapons  system  project,  in  terms  of  both  the  techniques  to  be  preferred  and  the 
sub-aystems  to  be  given  most  attention.  In  many  areas  of  airborne  weapons  system  technolojo-.  HAM 
activities  have  already  reached  a high  level  of  refinement  in  recent  projects.  The  UK  studies  mentioned 
earlier  (paragraph  5)  have  shown  that  good  returns  are  feasible  from  additional  RAM  investment  in 
future  projects  provided  that  the  choice  of  points  of  application,  and  of  the  most  appropriate  sub-system, 
is  shrewdly  made  during  all  the  phases  shown  in  Figire  1. 

71.  During  later  development,  transition  to  production,  early  production  and  early  service,  huge 
quantities  of  R A M data  become  available  in  an  airborne  weapons  system  project.  Their  acquisition  and 
processing  to  aid  timely  decisions  aimed  at  meeting,  or  revising,  RAM  objectives/ forecasts  poses  a 
considerable  challenge.  As  discussed  in  paras  21-25,  it  is  during  these  phases  that  the  main  threats  to 
reliability  and  maintainability,  and  specific  counter  measures  reveal  themselves,  or  are  confirmed. 

Together  with  th«  trade-off  analyses  of  para  69,  above,  the  data  acquisition  and  analysis  constitute  a 
need  for  a coherent  information  service. 


72.  Because  of  differences  in  required  capabilities,  in  technologies  available  at  t lie  time,  and  in 
ultimate  operational  usage. there  is  still  considerable  room  for  improving  the  bases  fort 

a.  setting  reasonable  targets  for  R A M at  the  outset  of  a project 

b.  comparing  the  RAM  achievements  at  the  various  stages  of  a project  with  those  from  earlier 
projects,  to  Indicate  the  progress  being  made  in  this  respect. 


75.  It  is  seen  that  the  control  of  life-cycle  costs  in  future  projects  is  likely  to  benefit  greatly  from 
improved  means  for  acquiring  and  interpreting  data.  The  main  requirements  for  analytical  support  (ie  a 
kind  of  "R  A M information  service")  for  the  management  of  a major  project,  in  these  respects,  are 
illustrated  in  Figire  12. 
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14.  Abstract 

I his  I ecture  Series  No. 100  on  the  subject  of  Methodology  for  Control  of  I ife  Cycle  Costs 
for  Avionics  Systems  is  sponsored  by  the  Avionics  Panel  of  ACARI)  and  implemented  by 
the  Consultant  and  Exchange  Programme. 

1 he  continually  increasing  costs  ol  avionics  systems  during  acquisition  anil  their  lifetime 
operation  is  a matter  of  grave  concern  to  the  NATO  family  of  nations.  I'he  N ATO  C.overn- 
ments  need  greater  visibility  and  control  over  the  life  cycle  costs  ol  any  weapon  or  avionic 
system. 


fortunately  there  have  been  tormulated  disciplined  methods  ol  providing  such  visibility  ami 
control  over  lile  cycle  costs;  that  is  over  the  development  acquisition,  training,  operating 
and  support  and  finally  disposal  costs. 

Il  is  proposed  that  the  Avionics  Panel  sponsor  a I ecture  Series  which  pa-sen ts  the  basic 
principles  of  Avionics  Systems  Cost  Analysis  in  a rapidly  changing  technology  environment 
and  gives  pnrn-n  methods  of  achieving  significant  costs  savings. 

I he  I ecture  Series  will  cover  the  following  subjects:  life  cycle  costing,  design  to  cost, 
technology  environment,  costing  of  software,  modeling  and  applications. 
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